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ABSTRACT 

This report presents the detailed results of the 
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discussions with experts, the information collection plan, and pilot 
test. Chapter III presents narrative summaries of the information 
compiled on each state, provided in alphabetical order by state name. 
The general format includes a description of the organization of 
vocational education in the state and estimates of the number of 
schools and students. Separate descriptions follow for public 
postsecondary vocational education, vocational-technical schools (if 
different policies and practices apply) , and proprietary 
institutions. For each component of postsecondary vocational 
education, the information assembled on the outcome measures 
collected, collection methods, data verification, data analysis, and 
uses are presented. Recently passes legislation and other indications 
of change in the policies and practices are also provided. Each 
discussion concludes with a section that highlights the most 
significant feature (s) related to the collection of outcome measures 
within the state. Chapter iv displays the key elements from the 
summaries in a matrix. (YLB) 
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STATE-LEVEL MEASUREMENT OF PERFORMANCE OUTCOMES 
IN POSTSECONOARY EDUCATION 

Vol use II: Appendix 
Profiles of State Data Collection Activities 

I. INTRODUCTION 

This report presents the detailed results of the compilation of an 
overview of state activities related to measuring the outcomes of 
postsecondary vocational education. An overview of the general study 
findings Is presented In Volume I: Executive Su mmary: An Overview 
n f State Policies . The effort was directed at several aspects of 
outcome measurement: 1) what measures were collected; 2) how the 
measures were defined and how the necessary data was collected; 3) 
what attempts were made to validate the meanres reported to the 
state; 4) what analyses of the data were performed at the state 
level; and 5) what uses were made of the measures, whether for 
management or for consumer rights, including reporting within the 
state education agency, reporting to the governor's office and 
legislature, reporting back to the schools, or reporting to the 
public. The charge was to provide a snapshot of the practices in 
place during the 1989-1990 school year nationwide. The document Is 
designed to familiarize policymakers with mere detailed reporting of 
study findings. The objectives are to provide descriptions, and not 
to provide evidence of the effectiveness of specific procedures and 
policies. However, where possible, this document highlights elements 
of the outcome measurement process that merit special attention in 
future policy considerations because of apparent threats to the 
validity of the measures, or because of problems in Implementation or 
acceptance. 
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Of the fifty states, District of Columbia, and Puerto Rico, no 
state 1 was found to have had an outcome measurement policy in place and 
unchanged for more than two years. The majority of states have either 
initiated new policies for collecting outcomes data, have expanded or 
improved their collection practices, have passed laws to make changes 
that will be implemented in the near future, or have introduced bills 
in the state legislature seeking authorization for similar changes. 

This report reveals that the states vary widely in the outcome 
measures they have selected to collect, the definitions of those 
measures, the method of collection, the analysis of these data, and the 
uses for which they are intended. Other than the pace of change and a 
consistent movement to increase the levels of activity in the 
collection of outcome measures, no patterns in the variation have been 
identified. 

Background 

Public interest in the issue of consumer information in post- 
secondary education, and particularly in postsecondary vocational/ 
technical (voc/tech) education, has increased over the last few years 
as a result of several converging influences. First, there is 
widespread concern over the quality of education provided by all types 
of educational institutions. Postsecondary vocational/technical 
education, like all other components of education in this country, is 
being asked to develop measures that demonstrate the quality of the 
education provided. Second, there is concern about access to a quality 

1 For convenience, throughout this report all of these 
jurisdictions are referred to as states. 



education for all segments of society. As the student bodies of 
voc/tech institutions come to have higher percentages of students who 
are from diverse ethnic backgrounds or are from disadvantaged 
backgrounds, there is concern that the education they receive be one 
that is likely to have returns to the individual and to society which 
are commensurate with the effort and cost associated with attendance. 
Third, with the rising cost of education, parents and students are 
increasingly interested in having evidence that the education they are 
purchasing is likely to result in program completion and employment. 
Finally, and most recently visible to the general public, there is the 
concern raised by the high rates of defaults on student lo^ns among 
students attending postsecondary vocational/technical institutions 
compared to other kinds of institutions. 

Each of these concerns results in an increasing interest in. the 
collection of data on the outcomes of educational programs, and in 
efforts to disseminate information about these measures to policymakers 
and to potential students and their parents in the form of consumer 
rights information. Research and Evaluation Associates, in conjunction 
with the Research Triangle Institute and the University of Maryland 
Baltimore County, was awarded a contract by the United States 
Department of Education to conduct four activities that constitute 
initial steps in the study of these issues. The first is the 
development of an overview of state activities in the collection and 
dissemination of outcome information. The second is a series of case 
studies to examine more closely the process of collection of 
information at the state level, as well as at the institutional level. 
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Included will be estimates of the level of effort required, practices 
that threaten the reliability of the data, and exploration of 
Institutional data collection practices Independent of any state or 
Federal requirements. The third will be a demonstration of the 
feasibility of using State Wage Data Information, submitted as part of 
each state's Unemployment Insurance System 2 , as a source of unbiased 
outcome Information for educational programs, and as a source of data 
that would require little effort on the part of Individual educational 
Institutions to collect. The fourth 1s a series of efforts designed to 
disseminate the findings of the overall study. 

Contents of This Report 
Volume I presents an overview of the study findings, including 
synopses of practices and major trends, as well as a summary and 
conclusions section. This report, Volume II, presents more detailed 
findings of an overview of state activities in the measurement of the 
outcomes of postsecondary education. The methodology used for the 
development of the overview is described in Chapter II. Narrative 
summaries of the information compiled on each state arc presented in 
Chapter III. The key elements from the summaries are displayed in the 
form of a matrix in Chapter IV. 



2 Hereafter in this paper this will be referred to as "state wage 
data." 
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II. METHODOLOGY 



Discussions with Experts 
As a first step in the design of the study, project staff held 
discussions with persons thought to be knowledgeable about the efforts 
to measure the outcomes of vocational and technical education by the 
states or by associations of trade and technical schools. The 
individuals and representatives of organizations with whom discussions 
were held are shown in Table II-l. The individuals were asked to 
describe the general trends in the employment of outcome measures and 
other efforts related to consumer rights information. The discussions 
also involved the identification of specific stdtes known by the expert 
to be actively involved in the outcome measurement process. The 
discussions concluded with a request for advice on how best to identify 
sources of information about the activities in each state. 

Information Collection Plan 
After the discussions with the national experts were completed, an 
information collection plan was prepared. After a review of the 
objectives of the overview, and based upon the information collected 
from the national experts and discussions among the project staff and 
the Contracting Officer's Technical Representative (COTrt), a list of 
information elements vas developed. The elements were chosen to 
represent the key descriptive information required to develop an 
understanding of the policies and practices of states in this area. A 
copy of the list is presented in Appendix A to this report. 




Table 1 1 - 1 . National Experts and Organizations Contacted. 



National Experts Contacted: 




John Lee 


Consultant to the Association of 
Independent Colleges and Schools 
and the National Association of 
Trade and Technical Schools. 


Richard Apling 


Congressional Research Service 


Darryl Parks 


Ohio State Department of Vocational 
Education 


Gene Bottoms 


Independent Consultant 


James Xadamus 


New York State Department of Educa- 
tion 


Dorothy Horell 


Red Rock Community College 


Gary Hoachl ar.der 
and Susan Choy 


National Center for Research on 
Vocational Education 


Organizations Contacted: 


Association of Independent Colleges and Schools 


National Association of Trade and Technical Schools 


American Council oi. Education - Center on Education and 
Training for Employment 


Alliance of Independent Colleges of Art 



A variety of methods was used to complete the information table 
for each state. The primary activity was the review of reports, 
legislation, and regulations from each state. As much of the table as 
possible was completed from these sources. After all written materials 
for a state had been exhausted, the table of information was reviewed, 
and areas where no information was available, or where clarification 
was needed, were identified. Project staff then made calls to 
officials from the state to fill in the information gaps or to provide 
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clarification. The information t..>ie itself was used to guide these 
discussions. No formal questionnaire was constructed and the questions 
asked of each official were tailored to match what was already known 
about the state and the official's relationship to postsecondary 
vocational education within the state. To complete the information, it 
was found necessary to talk not only to officials from the state 
departments of education, but also to representatives of the governor's 
office, the state legislature, the state loan guarantee agency, and to 
agencies responsible for the licensing and oversight of proprietary 
schools. 

Pilot Test 

To pilot test the information collection plan, senior project 
staff collected information from five states. Among these states were 
three nominated by the national experts as having active efforts to 
collect and disseminate performance measures, and '.wo nominated as 
having little activity in this area. Literature searches were 
conducted to identi.y information describing practices in these states. 
This was followed by calls to state public affairs offices to obtain 
reports and copies of legislation and regulations. These written 
materials were analyzed and the results entered into the table of 
information elements. Information gaps and needs for clarification 
were identified. State officials were then contacted to provide 
additional information. 

After the information collection process was completed, project 
staff developed prototype narrative summaries and an information 
matrix. The prototypes demonstrated that the useful information could 
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be collected and that analytical procedures could be developed to code 
and summarize the Information. 

The results of the pilot test led to some changes 1n the table of 
Information elements, particularly expanding the Information on how 
measures were defined, when procedures were Implemented, and how 
measures were used. The pilot test also demonstrated that one of the 
stiffest challenges during the full implementation of the data 
collection plan would be to identify which agencies within the state 
government are responsive for the various components of postsecondary 
vocational education in each state. No two states among the five 
states involved in the pilot test were organized in the same way. It 
was particularly difficult to identify the agency or agencies respon- 
sible for the oversight of proprietary schools. 

Full Implementation 

The full implementation of the data collection plan closely 
followed the activities of the pilot test. Ultimately, project staff 
were successful in completing the table of information elements for 
both public and proprietary vocational/technical education for all 
states but one. Only limited information from written sources was 
available to describe any measurement of the outcomes of vocational 
education in public community colleges in one state. 

The difficulty in identifying the agency(ies) responsible for 
oversight uf each component of postsecondary vocational program 
observed during the field test slowed the completion of the oversight. 
Multiple inquiries to state governments were required to develop a 
portrait of the organization of higher education in each statr\ In 

8 

ERiC 15 



many states, the project staff found great compartmentalization among 
the agencies involved. For example, those responsible for the 
community college system were unable to provide information about the 
oversight of vocational/technical schools or of proprietary schools, or 
even to provide the name of an individual or agency involved in their 
oversight. 

Limitations of the Findings 

Project staff often encountered inconsistent information about the 
policies and practices within individual states. It was not uncommon 
to find written materials that were not in agreement, or to find 
discrepancies between the information provided over the phone by an 
individual and information contained in a written source that 
individual provided to the project. As a general rule, written sources 
were accepted over the oral descriptions and precedence was given to 
the most current written document. These inconsistencies were 
especially disconcerting in those instances when two individuals in the 
same office of the same agency gave different information. For more 
than one state, an initial contact would indicate that the agency did 
not collect outcome measures and had no plans to do so. At a later 
date project staff would call back to the same office for 
clarification of a report. A different individual in the office would 
mention a new initiative in the development of outcome measures that 
built upon a ten year history of efforts in this area. 

Upon becoming aware of such inconsistencies, efforts were made to 
resolve them. The informant indicating that policies regarding outcome 
measures were in place was asked to provide copies of reports, 
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regulations or some other materials that supported that conclusion. In 
reviewing the results of this interview process, the reader must be 
aware of the possibility that in some states where project staff made 
only one phone contact an uninformed worker in the agency may have been 
the one to indicate whether written materials were available and to 
answer questions. 

Finally, the findings reported herein were collected primarily iti 
the early spring of 1990. As reported, these findings often were 
indicated to be in a state of flux. Therefore, they are reported here 
to represent a snapshot of the best efforts -« obtain the information 
at that point in time. 
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III. NARRATIVE SUMMARIES 

A major activity of the project during this task was the 
development of a descriptive summary for each state. The general 
format includes a description of the organization of vocational 
education in the state, and estimates of the number of schools and 
students. This is followed by separate descriptions for public 
postsecondary vocational education, vocational technical schools (if 
different policies and practices apply), and proprietary institutions. 
For each component of postsecondary vocational education, the 
information assembled on the outcome measures collected, collection 
methods, data verification, data analysis, and uses are presented. 
Recently passed legislation and other indications of a change in the 
policies and practices are also presented. The narratives vary fro;n 
state to state depending upon the organization of postsecondary 
vocational education in that state, the degree of activity in outcome 
measurement, the nature of the policies and practices in place, and the 
degree of similarity across the components of postsecondary vocational 
education. Each discussion is concluded with a section that highlights 
the most significant feature(s) related to the collection of outcome 
measures within each state. 

Few of the profiles present a separate discussion for the 
proprietary programs for the licensed trades that are overseen solely 
by the licensing boards. A sample of these boards was contacted in 
states that had this oversight pattern, but none were found that 
collected outcome measures other than success on licensing exams, and 
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none prepared reports that were used by other state agencies for policy 
planning or were disseminated to the general public. 



* 
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Alabama 



Alabama is comjpffing*<Jataba>? for piibUe s^te <biitalnu| enrollment numbers, major 
subject areas, and student Mo**etcnes, ttftfc wforawtfon on leavers ana completers to be 
added later. No pcffonnance measures are collected by the state for propria 



Organization of Postsecondarv Vocational Education 

The Department of Postsecondary Education, known as the Alabama 
College System, has authority over 39 schools offering vocational/ 
technical education: 20 technical colleges, 9 community colleges, and 
10 junior college*. These schools serve a total of approximately 
65,000 students. The Alabama College System is overseen by the Higher 
Education Commission which is responsible for budget recommendations, 
new programs reviews, and some planning. 

The Department of Education is the licensing authority for about 
300 proprietary schools. 

Public Schools 

The Department of Postsecondary Education began compiling a database 
during the past six to eight months, starting with enrollment numbers, 
major subject areas, and student biosketches. In the future, 
information on leavers and completers will be added. The department 
plans a year's experience with collecting the data before actually 
using it. The Higher Education Commission is considering some 
assessment measures but these would only apply to the four-year 
university system. 
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Proprietary Schools 

Proprietary schools must apply for a license on an annual basis. 
Every few years the state conducts a site visit which includes program 
review and facilities assessment. No measures of performance for these 
schools are collected by the state. 
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Alaska 



Alaska plans to institute a campus-based outcome assessment system for postseeondaty 
public schools funded by the state. Proprietary schools collect and report to the state their 
placement data by occupational area; they must provide supporting information to potential 
students when placement rates are used in advertising. The state is planning to institute a 
comprehensive statewide accountability assessment policy for nonpublic schools. 



Organization of Postsecondary Vocational Education 

Alaska's 17 community colleges were recently incorporated Into 
the University of Alaska System of Higher Education, which also 
contains 3 four-year universities with multiple campuses. The system 
is overseen by the Board of Regents. The nonpublic Institutions, 
overseen by the Alaska Commission on Postsecondary Education, consist 
of one junior college and about 77 proprietary schools including 
vocational/ technical institutes and trade schools. 

Public Schools 

Alaska has no statewide assessment program for postsecondary 
education. However, plans are underway to Institute a campus-based 
outcome assessment system implemented with state funding. Currently, 
some individual instructors wanting to keep track of their program 
graduates may collect completion rates, placement rates, success rates 
in career licensing tests, and relationship of placement to training. 
Completers are strictly the recipients of associate degrees. Completer 
data are compiled by gender and race. 
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The University of Alaska Statewide Institutional Research Division 
conducts a variety of studies on postsecondary education 1n the state. 
These studies are used as resources by the Governor and legislature in 
drafting appropriations legislation and for public policy decision 
making. One recent vocational education stuuy Investigated the reasons 
for a large number of the state's student loan borrowers choosing out- 
of-state vocational/technical education over equivalent training In the 
Alaska community colleges. 

Proprietary ScK ols 

The state has not required proprietary schools to submit 
enrollment data; the total number of students attending nonpublic 
schools 1s unknown. Placement data are collected by occupational area 
as part of the school annual financial report. Placement counts are 
based on the number of completers placed In any kind of job. Schools 
using placement data for advertising and other recruiting activities 
must submit placement rates and pertinent records by program and 
provide the Information to potential students on request. The 
placement rate 1s calculated as the percentage of completers placed 
compared to the number available for placement. Excluded from these 
counts are completers who are not seeking work, are pregnant or in poor 
health, are continuing their education, or who are financial debtors to 
the school. Placement data are collected on a standardized form 
prepared by the state. 

As a part of the license renewal process, site visits are 
conducted regularly. Institutional and program reviews are conducted 
every three years. New schools and new programs are reviewed every two 
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or three years. Placement data are reviewed every two or three years 
during site visits with additional, immediate site visits conducted 
whenever false advertising is identified. On occasion proprietary 
schools have been shut down for license violation, such as bond 
expiration, student complaints, or poor financial condition. 

In 1990, Alaska took steps to boost state authority over 
proprietary schools with priority being placed on greater school 
accountability. The Commission on Postsecondary Education is planning 
to develop a statewide accountability assessment policy for che 
nonpublic schools. It will take into account the various self -defined 
missions of participating institutions. Individual schools will use 
the statewide plan in developing student assessment policies relevant 
to their goals and objectives. Students are to be the primary 
beneficiaries of the quality assessment process. In addition, 
institutional accountability reports will be submitted annually to the 
Commission and made available to the public. These reports will 
include measures of performance such as entry, retention, and 
completion rates; placement rates; gains in basic and job skills; 
career licensing success rates; earnings levels; relationship of 
placement to training; employer satisfaction with graduates; student 
satisfaction with program; and minority recruitment, participation, and 
success. A formal definition of a completer is being developed. The 
methodology for collecting the performance measures will be determined 
In consultation with participating institutions. The Commission is 
also considering the requirement of student loan default rates for the 
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Institutional accountability report or as part of the license renewal 
process . 
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Arizona 



For public schools, Arizona annually compiles its Occupational information System (OIS) 
consisting of performance measures, inchjOing enrollment and completion figures, integrated 
wittt non-school provided information, such as employment and job opening, Job service 
and unemployment insurance data, grouped area. The Commission for 

Postsecondary Education has established an ad hoc committee to develop a statewide system 
for assessing institutionia eiTectivettess. 'Hm Board for Private Postsecondary Education 
collects data on enrollments, completers, and program offerings as part of the annual license 
renewal process. 



Organization of Postseco.Jarv Vocational Education 

Arizona has 16 public community colleges jointly overseen by the 
State Board of Directors for Community Colleges and the respective 
local districts. This Board also oversees one private community 
college operated by a Native American tribe. The 16 community colleges 
have 29 campuses organized in nine community college districts. 
Vocational and other programs are offered to the approximately 155,000 
students attending these schools, about 82 per cent of them on a part 
time basis. 

The State Board for Private Postsecondary Education regulates 145 
nonpublic vocational/technical institutions serving approximately 
106,000 students. Students in these schools are not eligible for state 
aid. Trade and industry schools in areas such as real estate, 
cosmetology, and barber training have statutory exemption from agency 
regulation. 

All performance outcomes collected by the respective education 
governing boards are submitted to the Commission for Postsecondary 
Education. The Commission compiles the data by vocational training 
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program and submits them to the Occupational Information System of 
Arizona (OIS) for analysis and dissemination. OIS Is a part of the 
state's Occupational Information Coordinating Committee and Is 
primarily funded by the Vocational Education Division of the state 
Department of Education. OIS Integrates the data from various training 
programs with non-school provided Information such as employment and 
job openings, job service and state wage data. The information is 
grouped by training area, summarized by the OIS cluster code, and 
published in the annual "OIS Cluster Summary Report" and several other 
specialized reports. The data are used in setting of training 
priorities and projecting placement opportunities and earnings 
potential . 

Public Schools 

The State Board for Community Colleges collects enrollment and 
completion figures, maintaining separate counts on minorities. A 
completer is the recipient of a diploma, certificate, or associate 
degree. Additional performance data may be collected by the community 
college districts but these vary across the state. The state Auditor 
General conducts on-site verification of enrollment data since 
funding 1s based on full-time equivalency enrollment. 

The Board for Community Education collects the enrollment and 
completion data annually in a paper format. The information is 
compiled by institution and training area and submitted in an annual 
report to the State Auditor-General and to the State Commission on 
Postsc-condary Education and the Occupational Information System of 
Arizona (OIS) for dissemination to the general public in the annual 
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Cluster Summary Report . The Governor and legislature receive copies of 
the report, which they use for appropriations decisions. 

The Commission for Postsecondary Education has established an ad 
hoc committee to develop a statewide system of assessing institutional 
effectiveness. The committee will conduct two sets of studies. The 
first group will consist of follow-up studies of community college 
graduates to track their placement, assess the relationship of 
placement to training, and document placement rates by institution and 
program. The second group of studies will look specifically at 
minority students co increase enrollment, retention, and placement of 
these students. One survey of minority completers will identify their 
initial intentions for enrolling in community college programs and 
whether or not their goals were met. A follow-up survey will determine 
their placement after completion. The ad hoc committee also is 
exploring the possibility of adapting Florida's performance outcomes 
model where data from the state Departments of Economic Security and 
Defense and the Internal Revenue Service would be ut.lized in the 
determination of placement rates. 

In 1985 the legislature sponsored a one-time study on the status 
of vocational education and institutional effectiveness for consumer 
rights purposes. The project, the "Study of Public and Private 
Postsecondary Institution Vocational Education Program Completers," was 
administered by Northern Arizona University's Arizona Center for 
Vocational Education. Graduates of community colleges and vocational 
schools received a questionnaire that asked about placement, training- 
placement relationship, and earning levels. The data were summarized 
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by institution type (public or private). The objective of the study 
was to enlighten the public on the quality of vocational programs 
available statewide. The Interim report released in June 1986 met with 
widespread dissatisfaction, especially from the State Department of 
Education, which generally had opposed the study from the beginning. 
Criticism focused on the study methodology, such as using too many 
variables and the inadequacy of the completer classification system. 
Consequently, the interim study report was never made public and the 
legislature discontinued the funding for the project. 

Existing legislation, "Goals for Education Excellence," mandates 
the three education beards —Commissi on on Higher Education, the 
Community Colleges Board, and the Board of Education (K-12)--to develop 
jointly a longitudinal student tracking system to monitor student 
progress statewide from kindergarten to the university level. However, 
state department of education staff indicated the law has never been 
implemented. 

Proprietary Schools 

The Board for Private Postsecondary Education collects data on 
enrollment, number of completers, and programs offered as part of the 
annual license renewal process. A completer is anyone awarded a 
certificate or diploma. Completion and placement rates, earnings, and 
other measures of performance are not required for licensure. Annual 
site visits are conducted in each school for product and financial 
audits. Student records and the condition of the facilities and 
equipment are routinely reviewed to assure compliance with statutory 
license requirements. Spot checks of submitted information also are 
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conducted to verify the accuracy of data submissions. The greatest 
number of Inaccuracies are found In financial documentation. However, 
compliance with information requirements is generally high, and schools 
are mostly cooperative with the review effort. 

When public complaints are filed against a school, a site visit is 
made to conduct a program review. To conduct such investigations, 
regulatory provisions give the Board access to a variety of non-school 
data sources such as credit bureaus and other regulatory bodies with 
jurisdiction over participating schools. 

The school -provided data are strictly for internal management 
decisions. However, the data are ultimately forwarded to the 
Occupational Information System for publication in the annual QI& 
Cluster Summary Report and the Arizona Proprietary Schools and 
Community Based Organizations Enrollments and Completers . In the 
latter report the information is compiled by vocational training areas, 
program, and county. 

The Governor reportedly is committed to enhancing the quality of 
education obtained at vocational/technical proprietary schools. She is 
playing an active role in pending legislation seeking a special study 
to investigate the potential for the transferability of academic credit 
freely among public and private universities, community colleges, and 
vocational/technical institutes. Current procedure gives the 
universities the discretion over acceptance or rejection of transfer 
credit. Institutional negotiations are underway to identify 
transferable community college credits and the university core programs 
1n which they would fit. The Governor reportedly also would like to 
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reimplement the 1986 consumer rights follow-up study of vocational 
program completers. 

Pending legislation for elementary and secondary education would 
set statewide academic goals and attempt to measure effectiveness in 
the work place and at higher education levels. This bill has potential 
impact on postsecondary vocational/technical education. 
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Arkansas 



Using on-line data, Arkansas collects student completkin and placement data as well ^ 
Initial jot> wag* rates for jroutie ri&i^ A new system vtft track 

all students of two-year public Institutions ol graduation and retention rates. No data are 
coUected from proprietary scho^ : !SS 5: :;:lil v; ::" 



Organization of Postsecondarv Vocational Education 

The Department of Vocational and Technical Education oversees 24 
vocational technical schools and has the authority to approve the 
curriculum of vocational programs at the community college level. The 
State Board of Higher Education oversees the two-year institutions, 
which consist of eight community colleges and three branch campuses of 
universities. 

In September 1989, the proprietary schools were removed from the 
Department of Vocational and Technical Education and placed under a 
separate, newly created State Board of Private Career Education. The 
schools were said to endorse this new arrangement, which is expected to 
provide closer monitoring. 

Public Schools 

Vocational -Technical Schools . The state collects completion rate 
data from the vocational/technical schools. A completer is defined as 
a full -time student who finishes all program requirements. Placement 
rates also are collected based on the percentage of completers who are 
in jobs related to their training. The school superintendent of 
instruction decides if a match exists between a student's program and 
subsequent job. Initial wage rates also are collected. The Department 
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of Vocational and Technical Education receives passage rates on career 
licensing tests from the respective boards. 

The schools are on-line, so transfer of Information to the state 
Is rapid. A concentrated effort is made to keep enrollments and 
placements hig*\ If questions about data arise, the state will contact 
the school, usually by telephone. 

An additional source of data is the one-year follow-up survey of 
students who have completed programs. The survey is mailed by the 
Department of Vocational and Technical Education and respondents return 
the information directly to the Department. Questions about placement 
and wages are Included on the survey. 

Many requests for the data are made by the Governor and 
legislature. The schools use the data for program planning. Whole 
programs have been dropped and added in the last few years. Disclosure 
to students and the general public is made at the discretion cf the 
schools. 

The Department also audits student loan default rates. Schools 
must tell what they are doing to address problem areas. Information is 
collected on school financial condition and facilities and equipment. 

Two-Vear Institutions . In 1989, the legislature passed a bill 
that mandated the development of a tracking system for all students in 
state sieges and universities. This system will enable the state to 
calculate retention and graduation rates. The first report using this 
system is due in fall 1991. 
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The state does not collect Information on placement of graduates, 
initial earnings, or success rates on career licensing tests from the 
schools. 

Proprietary Schools 

At the current time, no data cn performance outcomes are collected 

from the proprietary schools. 
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wmance measure collected tor public schools in California is number of school 
t ... Nfflieasures of performance wera collected from propriewfyafcc^how^r, 
the Marine Waters School Reform and Student Protection Act of 1989 goes into effect in 
1991 squiring programs of less than two years duration in these institutions to report 
completion, job placement, and licensure/certification rates and, Af used for promotion, 
evidence of salary claims. ., /?V-*-.-< 



Organization of Post secondarv Vocational Education 

California has 107 community colleges serving approximately 
1.4 million students. Over 700,000 of them are enrolled in vocational 
education programs. Regional Occupational Centers are the state's 
equivalent of postsecondary vocational/technical institutes. They are 
operated by high schools but offer specialized vocational programs for 
adults as well. 

Specialized career-oriented institutions falling under the 
jurisdiction of the Department of Consumer Affairs are reviewed by 
their respective boards; these include schools of barbering, 
cosmetology, acupuncture, and vocational nursing. The Private 
Postsecondary Education Division of the State Department of Education 
oversees the remaining 2,200 proprietary schools, which enroll almost 
1 million vocational students. 

Public Schools 

Information on student completers is the only performance measure 
collected at this time. There are no plans presently to collect 
additional measures. 
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Proprietary Schools 



At the present time, no measures of performance are collected from 
proprietary schools. Instead, the state relies on the judgments of all 
federally recognized accrediting agencies to monitor the quality of the 
postsecondary institutions and their programs. 

The Maxine Waters School Reform and Student Protection Act of 

1989, which goes into effect in January 1991, contains new requirements 

for proprietary postsecondary schools with programs that are two years 

or less in length. Schools that have been in operation for at least 

one year will be required to provide the following information to the 

state and to prospective students before they sign agreements 

obligating them to pay any money to an institution: 

1) The completion rate for students in the course of instruction 
for the most recent calendar year that ended not less than 18 
months prior to the date of disclosure. This rate is to be 
calculated by determining the percentage of students enrolled 
in the course of instruction who were originally scheduled, 
at the time of enrollment, to complete the course of 
instruction in that calendar year that successfully completed 
the course of instruction, or obtained full-time employment 
in the occupation or job title category for which training 
was offered, within 150 percent of the amount of time 
normally required to complete the course of instruction. 

2) The job placement rate for students who were originally 
scheduled, at the time of enrollment, to complete the course 
of instruction in the most recent calendar year that ended 
not less than 18 months prior to the date of disclosure. In 
calculating this rate, the institution is to consider as not 
having obtained employment any graduate or other student for 
whom the institution does not possess evidence, documented in 
the student's file, showing that the graduate or other 
student has obtained employment in the occupation or job 
title category for which the course of instruction is 
offered . 

3) If the institution or representative of the institution makes 
any express or implied claim about the salary that may be 
earned after completing a course of instruction, the school 
must provide the percentage of students who were originally 
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scheduled, at the time of enrollment, to complete the course 
of instruction 1n the most recent calendar year that ended 
not less than six months prior to the date of disclosure who 
earn salaries at i»r above the claimed level and the ranges of 
monthly salaries earned by these students in $200 increments 
and the number of these students In each salary range. 

4) If the course of Instruction leads to employment 1n a field 
requiring a state licensing examination, the school must 
provide the licensure or certification requirements and the 
pass rate of graduates on the licensure or certification 
examination. 

To obtain placement information, schools must conduct a six-month 
follow up of graduates to determine which ones have obtained employment 
in the occupation in which they received training. The schools must 
obtain the name, title, or description of the job; the date the student 
obtained employment; and the employer's name and address. 

An Institution is required to discontinue offering a course of 
instruction if more than 40 percent of the students fail to complete it 
or If more than 30 percent of the completers fail to obtain employment 
within six months. Schools may seek to exclude specific students from 
the calculation of placement rate. 
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Colorado 




Organization of Postsecondarv Vocational Education 

The Colorado Commission on Higher Education generally oversees the 
state's public postsecondary education. However, the Sta Board for 
Community Colleges and Occupational Education Is the day-to-day 
oversight agency for community colleges and postsecondary vocational/ 
technical schools, both public and nonpublic. 

Public Institutions offering postsecondary vocational programs 
Include 15 community colleges, 2 four-year colleges with approved two- 
year vocational programs, and 7 area vocational schools offering 
programs at secondary and postsecondary levels. The Board serves 
approximately 69,000 students, of whom 9,000 are In area vocational 
schools and 60,000 In the community colleges. Four of the community 
colleges are organized under local districts whose boards of trustees 
oversee their academic track programs. Pending legislation wc 14 grant 
autonomy to the area vocational schools and transfer any remaining 
oversight responsibility to the state Department of Education. 

The Board for Community Colleges and Occupational Education has 
oversight powers, through licensure, over the state's 58 proprietary 
schools that serve approximately 22,000 students. 
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Public Schools 

Recent legislation established the higher education accountability 
program requiring all public postsecondary Institutions except Area 
Vocational Schools to submit annual self -account ability reports to 
pertinent oversight agencies starting in October 1989. The legislation 
focussed on the assessment of student outcomes. Community colleges 
submit data on enrollment, completion, placement, relationship of 
placement to training, gains in basic and job skills, and success rates 
in career licensing exams. They also must report Institutional changes 
made or planned as a result of the school self -assessment. The reports 
must incorporate a survey of students' long-term goals for attending 
the chosen institution. The Board maintains all the Information in a 
central data base system, hence data are mostly submitted on computer 
diskettes. 

Enrollment and completion data are collected on semester and annual 
basis respectively. The State Board is in the process of developing a 
standard definition of a completer and a formula for computing 
completion rates. Generally, the institutions define a completer as 
the recipient of a certificate, diploma, or associate degree. Data on 
completers is subdivided by gender, racial/ethnic category, and 
handicap. 

Placement information must be obtained through a completer follow-up 
survey conducted during November through March after graduation. Each 
school develops its own survey instrument, which is administered by the 
vocational Instructors. Respondents are asked about their currently 
positions, such as continuing education, employed In training-related 
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or unrelated jobs, or unemployed. Earnings Information Is not a 
reporting requirement, but the oversight agency has access to state 
wage data and frequently utilizes It for Institutional analyses. 

Assessing gains In basic or job skills Is the core of the 
accountability legislation. The higher education accountability 
process Is very focussed on "value added" outcomes, requiring 
Institutions to show evidence of skills improvement. Tests are 
therefore administered at entrance 2nd exit to assess gains 1n both 
basic and job skills. Assessment Instruments Include the ACT and 
program-related exams. The Individual student Is the unit of analysis. 

All of the Institutions undergo financial and administrative audits 
through annual site visits. Moreover, the Board's central data base 
system facilitates year-round monitoring of data accuracy and 
compliance with reporting requirements. Although compliance with the 
reporting requirements was said to be good, to Increase data accuracy 
there are tentative plans to limit Institutional collection of certain 
types of the follow-up data (e.g., training-related placement, wage 
data) to only those completers the oversight agency cannot locate. 
Pilot studies show that two-thirds of the alumni can be located up to 
two years after graduation. 

The data collected are mostly for Institutional analysis and program 
evaluation. The analyse* and evaluative Interpretation of the 
Information are widely disseminated through summary reports distributed 
to the general public for consumer rights purposes, to the Commission 
for Higher Education for management decisions, to the Governor and 
legislature for law and public policy-making, and to the business 
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sector for placement and other needs. The reports Include: 

o Community Colleges Fact Book , directed to the general public. 

o Annual Report, prepared under the State Vocational Act, which 
gives information on vocational programs available statewide, 
the number of completers by program, etc. for public use. 

o Connections, a bimonthly newsletter designed for the private 
industry. 

o Accountability reports from each public institution, which go to 
the Commission for Higher Education, the Governor and the 
legislature, and are available to the public on request. 

New measures of performance being considered include revival of the 
continuance-discontinuance model of program analysis and the 
development of a standardized instrument to assess alumni employment 
satisfaction. In the former, a program is divided into quintiles and 
examined in detail for effectiveness and viability. The revived model 
would take into account any lost data, the efficiency and inefficiency 
issue, and other factors bearing on program feasibility. The proposed 
alumni study would survey vocational education completers and non- 
completers, and academic track continuing education students on their 
satisfaction with their current status. Employment satisfaction would 
be assessed as applicable. 

Proprietary Schools 

Completion and placement rates are the only measures of performance 
collected from proprietary schools, although the state has the capacity 
to collect others through the central data base system. A completer is 
defined as one who has passed all required courses of a program and was 
awarded some kind of completion document. The completion rate is the 
percentage ratio of the total number of completers to the total 
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enrolled, broken down by gender and program. The placement rate 1s 
expressed as the ratio of the total number of graduates employed to the 
total who completed. Both measures are standardized, but not used as 
a condition for licensing. Consequently, no effort 1s made to verify 
accuracy of reported data. 

Data are collected annually on program-specific summary forms 
prepared by the oversight agency. The information is used strictly for 
management purposes, particularly as a basis for license renewals and 
determination of surety bond levels. Annual site visits are made to 
assure compliance with licensing requirements. Several schools have 
been shut down for license violations, such as failure to provide the 
education Intended, inadequate financial condition, unqualified 
faculty, and faulty equipment. The agency is currently in litigation 
with several other schools facing closure. A key defense of the 
schools 1s that the state wants to eliminate them for the stiff 
competition they pose to state-funded vocational/technical programs. 
Overall, the relationship between the oversight agency and 
participating schools is characterized as "businesslike," although some 
schools cited for noncompliance reportedly are hostile. 

Newly passed legislation effective In fall of 1990 created the 
Colorado Private Occupational School Authority and will transfer all 
oversight of postsecondary proprietary schools to a specific division 
in the Department of Higher Education. This reorganization is expected 
to Increase state ability to regulate these schools. Sponsored jointly 
by the Board for Community Colleges and Occupational Education and the 
Colorado Proprietary Schools Association, the legislation is also 
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expected to make the litigation process for school closure more 
efficient. 
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Connecticut 



In Connecticut* demographic, retention, basic skills, and career licensing data and number 
(but not rates) of completers and placements are cotiecied ^ 
technical colleges, with data available by gender and racial/ethnic status. Graduate surveys 
coUea informs 
of twtiring^ 
^Hc vocaflon^ 
' ;yV ' propriety 



Organization of Postsecondarv Vocational Education 

Connecticut has 5 technical colleges that enroll about 5,000 
students, all of them in vocational areas. The 12 community colleges 
in the state have a total of approximately 36,000 students. These 
schools are overseen by the Board of Trustees of Community-Technical 
Colleges. 

The vocational technical school system consists of 17 schools, 
which are totally run and operated by the state. Most of the students, 
about 11,000, are at the secondary level. Approximately 1,000 students 
are post-graduates in certificate programs. An additional 5,000 
students are in apprenticeship programs. 

The Vocational, Technical and Adult Education Division of the 
Department of Education oversees the approximately 80 proprietary 
schools in the state. The total number of students enrolled in these 
schools is unknown; there is a debate about whether or not the state 
should gather this information. 
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Public Schools 

Community and Technical Colleges . For community and technical 
college programs, individual student tracking is done at the state 
level. A considerable amount of demographic and retention information 
is maintained. Many of these data are collected at the time of 
registration. At some colleges, the students fill out computer cards; 
at other colleges, the students write out the information, which is 
then entered by the school into an on-line computer. 

The state does not calculate completion rates but has the 
capability of doing so. Completion information is available; a 
completer is one who finishes a program of study and receives a degree 
or certificate. Data are available by gender and by racial/ethnic 
status. 

It was estimated that 70 percent of the students in the technical 
and community colleges already are employed while attending school. 
The state does collect information on the number of placements but not 
on placement rates, "he colleges send a survey to all graduates six 
months after graduation. The forms are returned to the colleges, which 
forward them to the state- for analysis. The system has the capacity of 
providing placement information by racial/ethnic status, gender, and 
age but does not currently present data in this way. 

The graduate survey is updated annually. In the survey of 1987-88 
graduates, respondents had to select from the following categories for 
occupational status: full-time military, full-time employment, part- 
time employment, unemployed but seeking employment, and unemployed and 
not seeking employment. Graduates who were unemployed and not seeking 
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employment had to explain why. If employment was not related to the 
training program, the graduates had to tell why. Respondents were 
asked 1f their training program helped them 1n obtaining a Job or 
getting a salary Increase or promotion. Graduates also rated their 
training programs on a number of factors. 

Most students In the community and technical colleges are given 
the New Jersey Test of Basic Skills at two different times during their 
postsecondary programs. Reading, language, and mathematics are the 
areas assessed. Scores are reported to the state. The state also 
collects success rates on career licensing tests when applicable. 

Twenty percent of the programs in each college are reviewed every 
year. The schools must conduct a self-study of each program being 
reviewed that includes statistical information for the three years 
preceding the evaluation. Among the items are number of enrollments, 
graduates, and placements by year, and number of graduates successfully 
meeting licensure/cortification requirements by year. Surveys ef 
graduate and employer reactions also are conducted. Following the 
self-study, an evaluation team that reflects the expertise of 
employers, graduates, and specialists makes a site visit to the school. 
The goal of an evaluation team 1s to compare the self- study report 
against the stated objectives of the program being reviewed. The 
report of the evaluation team is submitted to school administrators and 
trustees but not to the state. The state does not do a review of the 
student data. 

An annual report on the technical and community colleges is 
prepared for the State Department of Education. An enrollment report 
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and a graduate survey report also are done annually. Information from 
these reports 1s used by the postsecondary schools, secondary schools, 
the Governor, and the legislature. The primary use of the data is for 
management decisions. The reports are public documents and students 
have access to them at libraries but the reports are not sent to 
students. 

Vocational Technical Schools . Enrollment and completion 
information is available by program in the vocational technical 
schools. Some programs have both secondary and postsecondary students; 
however, reporting is not done separately for these two groups. 
System-wide data are available on basic skills. Competency skills by 
trade also are utilized. A graduate follow-up is done only for 
secondary students. 

Proprietary Schools 

Proprietary schools apply for state approval every three years. 
The applications include information on size and claims toward 
placement. The data are not collected in any uniform way or reported 
as such. The application for state approval also must contain 
information on the curriculum, faculty, facilities, financial status, 
and success rates on career licensing tests. 

No separate reports are done on completion or placement. 
Performance measures that are being considered are enrollments, 
completion, dropouts, and placement. 
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Delaware 



Delaware currently docs not collect performance measures for postsecondary vocational 
education. The Governor is receptive to establishing a state policy for outcome measures, 
but thjs is not indicated to be a priority for the legislature. 



Organization of Postse condarv Vocational Education 

The Postsecondary Education Commission in Delaware oversee?; one 
community college with four campuses, 4 private four-year colleger, and 
2 public four-year colleges. Altogether, about 40,000 students attend 
these schools. The Private, Business and Trade Schools unit in the 
Department of Public Instruction oversees 30 proprietary schools. 

Public Schools 

Delaware has no state policy regarding outcome measures. The 
Division of Academic Program Review publishes the Annual Data Book on 
postsecondary institutions. It contains the total enrollment for each 
institution with breakdowns by sex and full -time/part-time students. 
Number of degrees awarded by field of study for the state as a whole is 
also presented; subdivisions by race and sex are included. 

Proprietary Schools 

No performance measures are collected at the state level and none 
are being considered. The state has few, if any, complaints about 
approved proprietary schools. 

Annually, proprietary schools must submit application forms for 
state approval. The applications contain information such as school 
financial condition, admission and refund policies, equipment and 
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facilities, and faculty qualifications. Representatives of the state 
visit all schools annually to verify the infor "tion submitted in the 
application approval package. Violations of approved requirements have 
on occasion led to program or school closure. 

A directory of approved proprietary schools is published annually 
and made available to the general public. Schools and programs that 
fail the annual approval are excluded from the directory. 
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District of Columbia 



the District of Columbia doe* sot 
and, at present, does not collect 




vocational education 



Organization of Postsecondarv Vocational Education 

The District of Columbia does not provide :ny public postsecondary 
vocational education. In 1989, the Education Licensure Commission was 
given the responsibility to oversee proprietary postsecondary schools, 
which include 1 vocational college and 39 nondegree postsecondary 
schools. 



Proprietary Schools 

At present no data are collected from these schools but completion 
and placement rates are under consideration. 
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Florida 



Bdrida began the Occupational Identifier Project te 1984 to establish an effective tppioadi 
to collect Mow-up data oh former public school ^tiomrt suideiits. >to# i»^ 
records from nine different data files, this system provides outcome data on job placement, 
employment earnings, outcome data on job placement, employment earning* continued 
schooling, and relationship of employment to training. Established as an ongoing program 
in 1988, it has been expanded to include swdent and employer opinions. In 19^ a y 
productivity standard was mandated based on completion rates. Requirements for 
proprietary schools are moving towards meeting the requirements for public school?. 



Organization of Postsecondary Vocational Education 

The public postsecondary vocational/technical schools in Florida 
are administered at the state level by the Division of Vocational, 
Adult and Community Education within the Department of Education. The 
schools are organized in 67 districts throughout the state. The 
nonpublic schools are monitored at the state level by the State Board 
of Independent, Postsecondary, Vocational, Technical, Trade and 
Business Schools. The board oversees about 600 schools. 
Responsibility for 150 schools of cosmetology, barbering and truck 
driving were added at the end of 1989. 

Public Schools 

In recent years, significant steps have been taken to measure 
placement success and to tie performance on this measure to funding. 
In 1984, the Florida legislature initiated the Occupational Identifier 
Project, which sought to determine an effective approach of collecting 
follow-up data on former vocational students. The strategy used by the 
project was to conduct a computer match of individual student records 
with the following data bases: the state's quarterly Unemployment 
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Insurance Employer Wage and Tax Report (state wage data), the U.S. 
Department of Defense's Manpower Enlistment Files, the U.S. 
Government's Personnel Management Information System, the U.S. Post 
Office's Personnel Management System, the State Department of 
Administration's State Employee Files, the State University System's 
spring and initial fall enrollment files, and the Community College's 
fall enrollment files. These computer matches provided information on 
job placement, employment earnings, and continued schooling in 
Florida's public community colleges and universities. In addition to 
the computer match, the state Department of Education sent Florida 
employers of former students a brief form on which they entered the 
occupations and employment locations of the former students; occupation 
title was needed to determine the relationship of placement to 
training. 

In 1988, the legislature required that the project be established 
as an ongoing program; it was renamed as the Florida Education and 
Training Placement Information Program. New components continue to be 
added to the program. In 1990, two opinion surveys were conducted. In 
one of them, a sample of employers known to have hired former students 
during the previous year was asked about the preparation of the 
students for employment. The second survey, sent to all Florida 
employers who hired licensed practical and registered nurse graduates 
during the previous year, focused on employer satisfaction. 

In 1989, the legislature mandated a productivity standard based on 
completion rates, defined as the comparison of the number of students 
enrolling in a program and the number completing. An improvement of 
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twenty percent must be shown each year. A task force was given one 
year to establish standards across the state. One factor 1n the 
development of a placement standard being considered 1s a result of a 
study being conducted by Florida State University of the reasons 
students drop out of vocational/ technical programs. Preliminary 
results indicate that the programs do not have enough exit points 
labeled as completions for students who enter these programs with 
different goals. This may change the way a program completer 1s 
defined. 

Considerable attention also is given to meeting a state mandated 
placement standard. If a program's placement rate falls below 70 
percent for three consecutive years, the program loses state funding. 
In earlier years, the schools designed and conducted their own surveys 
to provide placement Information. Starting in 1989, the calculation of 
a placement rate for a school or program is being made using state wage 
data records. Whether a placement 1s training related 1s determined by 
comparing the industry code of the employer to the Classification of 
Instructional Program (CIP) codes for the student. 

A report on success rates in career licensing tests also must be 
made to the state board and legislature. The report includes pass/ fall 
rates per region, which are reported for 16 of the 22 licensed 
occupations for which arrangements have been made to track exam 
requests by school. 

The state is working toward on-line student reporting. A common 
database from the Division of Community Colleges, public schools, and 
the Department of Labor would be developed. It would be possible with 
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such a system tc retrieve data on Individual students or categories of 
students at any point 1n their careers and to assess the relationship 
between the educational program and subsequent employment. 

The Information currently reported to the state by the Individual 
schools 1s presented to the state legislature. The basic organization 
of the report 1s by planning region, of wh ch there are 28 In the 
state. The report also provides Information by school and by p jgram. 
Tho report for each school year 1s Issued the following spring. These 
reports are based on Information for the first quarter after graduation 
for May/June graduates. The state recognizes that this timing has the 
disadvantage of Including many summer-only jobs as employment. 

Copies of the reports are sent to school districts and to the 
community colleges. Disclosure to students or prospective students Is 
not required. The uses of the Information In marketing are at the 
discretion of the Individual schools. 

The rate of compliance with the requirements 1s currently quite 
high with complete Information being submitted. Two years ago, six 
districts had their Federal funding terminated because of 
noncompliance, and three other districts were threatened last year. 
This appears to be a major motivating factor In the prompt completion 
of reports. 

Program review Is completed once every five years, and is 
conducted on-site by state monitoring teams. Also, teams of auditors 
from the Division of Administration in the Department of Education 
check the records against the Information reported. The accuracy of 
the reported Information has been high. 
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Proprietary Schools 

Requirements for the proprietary schools are said to be moving 
toward meeting the public requirements. Currently, each year the 
private schools submit the number of students enrolled, withdrawals, 
dismissals, re-enrollments, and graduates by program. Schools also 
must send placement data based on a survey of a sample of students and 
their employers. The employers make the determination of whether the 
job 1s related to training. 

The data are submitted on a standardized form prepared by the 
state. The Information must be Included 1n the applications for 
renewal, so the rate of compliance Is high. The reported data are 
audited during on-site visits made to the schools on the average of 
once per year. The state also calls a sample of the students surveyed 
to verify the Information submitted by the schools. The reported 
Information has generally been found to be accurate. 

The Information on the private schools 1s disseminated in the form 
of a booklet called "Choices," which Is distributed to high schools. 
The booklet presents the Information by proprietary school and by 
program within school. The reported Information also 1s reviewed by 
the state board, and the dropout and placement rates are examined. The 
board may request a placement Improvement plan or a retention 
Improvement plan, which may result in changes 1n or discontinuation of 
a program. 




^Miej^ts^ 
j^pce^tttri^ 

surveys, -am each related to the field of study. No performance measures are collected 
front proprietary schools. 



Organization of Postsecondary Vocational Education 

The Technical and Adult Education Department is responsible for 
public postsecondary vocational education in Georgia. The department 
oversees 32 postsecondary vocational/technical schools, which enrolled 
25,658 students in FY 1989. At one time these schools were treated as 
an extension of the secondary system, but they were moved into a 
« separate department duringihe early 1980s. With the exception of some 
health occupational programs, the 15 two-year colleges in Georgia are 
not vocationally oriented. The focus of these colleges is to prepare 
students for eventual transfer to the four-year colleges and 
universities. 

The 121 licensed proprietary schools currently are overseen by the 
Department of Education. Legislation is pending that will p T ace these 
schools under a separate commission and strengthen standards and 
regulations. 

According to the Governor's assistant on the Council on Vocational 
Education, the council acts in an advisory capacity only. At present, 
thr council is reviewing Florida's program. Most assessments in 
Georgia are currently conducted as self -analyses at the department 
level . 
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Public Schools 

Completion rates are calculated for vocational/technical programs. 
Completers are defined as those who have finished a certain percentage 
of the program requirements (the percentage varies by program) and been 
placed In a related job; only full-time students are Included In the 
calculation. Separate completion rates are calculated for women and 
minorities. The numbers of handicapped and disadvantaged students in 
the total enrollment 1s collected but separate completion rates are not 
calculated for them. 

Placement rates of students in jobs related to their field of 
study are also calculated. The Information 1s obtained from student 
surveys. The placement coordinator at each school 1s responsible for 
conducting the surveys and determining if the job 1s related to the 
field of study. Separate rates are calculated for minorities and 
women. 

Earnings Information is collected through Initial and one-year 
follow-up surveys. Earnings are reported by major subject area. 

Currently the schools provide the data to the state both quarterly 
and annually. A new system is being installed that will enable the 
state to be on-line with all schools. Reviews are done when the state 
has questions about the data. The review may be done by on-site visits 
or by telephone. The level of compliance and the accuracy of the data 
were said to be good. 

The information is used in a statewide evaluation of training and 
placement effectiveness of the schools. It is also used in reports 
that are sent to the state advisory council, the Governor, the 
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legislature, and the schools. The data are available In each school's 
placement office where 1t can be reviewed by students, potential 
students, and the general public. 

Postsecondary schools use the data for planning purposes and 
making management decisions. Program changes are common; programs have 
been started, terminated, increased, and decreased as a result of the 
data. 

Proprietary Schools 

Licensing standards for proprietary schools are minimal and no 
measures of performance are collected from them. 
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H awaii 

Hawaii collects performance outcome data from public schools as part of its five-year self* 
stu#^ 

program area, include completion, placement, current earnings levels, relationship of 
placement to training, and success rates on licensing exams (for nursing). No performance 
measures are collected from proprietary schools. 



Organization of Postsecondarv Vocational Education 

The public postsecondary schools in Hawaii consist of 2 four-year 
colleges and 7 community colleges administered under the University of 
Hawaii System. All public postsecondary vocational education is 
delivered through the community colleges. There are no public 
postsecondary vocational/technical schools. The state's seven 
community colleges are incorporated in the University of Hawaii system 
and serve an estimated 23,0C0 students. 

The nonpublic schools include 5 four-year private colleges and 35 
proprietary schools (10 vocational -technical institutes and 25 trade 
schools). The State Department of Education annually licenses the 
proprietary schools, which serve an estimated 1,046 students. By state 
law, proprietary students are not eligible for state financial aid. 
Schools in fields such as cosmetology and real estate are overseen by 
their respective governing boards. 

Public Schools 

Performance outcome information is collected from each of the 
sev n community colleges as part of the five-year self-study process 
required for accreditation purposes. Outcome data are primarily used 
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to compile required program evaluation reports. Plans are underway to 
enhance the program evaluation process with longitudinal follow-up 
studies of graduates and those who left their programs. Statewide 
program evaluation ended with the Perkins Act. 

Performance outcome information collected includes data on 
completion, placement, current earnings levels, relationship of 
placement to training, and success rate on the nursing career licensing 
test. A completer is defined as either anyone awarded an associate's 
degree, diploma or certificate, or a partial program completer who 
successfully attained training-related employment. Placement data are 
generated through graduate and leaver surveys conducted five years 
after departure from school. Survey questionnaires and follow-up 
telephone interviews are used to identify the number of individuals 
placed, earnings levels, and the relationship of placement to training. 
The percentage employed in training-related jobs is a vital data 
element in the program evaluation requirement. 

For each campus, information is organized by student and program 
or major area of study. The five-year program evaluation report is 
submitted in a paper format. Site visits are conducted jointly with 
the office of the State Director for Vocational Education to evaluate 
the program review process. Compliance with information requirements 
data accuracy rate were reported to be good. Data from the schools and 
other sources such as labor market data ard pertinent studies are used 
to projt ~* the continued need for a program. At least one program a 
y*ar has been shut down or initiated on the basis of this information. 
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The program evaluation efforts are aimed primarily at facilitating 
management decisions to improve program quality. 

Collected information on enrollment and completion, programs 
offered, program reviews, etc., 1s disseminated in an annual evaluation 
report of the community colleges distriouted to the State Director of 
Vocational Education and the general public. A summary of program 
evaluation and reviews goes to the President of the University of 
Hawaii System for use in his report to the State Board of Regents. The 
President's evaluative reports are often used in legislative funding 
decisions. Numerous other evaluation documents are produced annually 
for internal purposes. 

The state has no pending or proposed legislation addressing the 
issue of consumer rights information in postsecondary education. 
However, the State Department of Education has entered a request in the 
appropriations bill seeking an increase in its operations budget to 
help fund data collection processes for statewide performance outcome 
assessment. 

Proprietary Schools 

The state collects no measures of performance or any outcome- 
related data from the proprietary schools, as these are not required 
for licensure. Site visits are made only to investigate filed student 
complaints, such as school failure to deliver instruction as advertised 
in a brochure or the withholding of tuition refunds. Most complaints 
are resolved outside litigation. School closure as a punitive measure 
is rare. The state publishes an annual directory of licensed 
proprietary schools, which is distributed widely to the public. 



The measurement of the outcomes of vocational education in Idaho is concentrated on the 
public schools and little attention is given to the proprietary schools. Measures collected 
and reported for public schools include completers, employment status, relationship of 

: ^training to jtib' placement, and Initial wages. These measures are reported by s$to! and by 

•' /program •" 



Organization of Postsecondarv Vocational Education 

Idaho is a small state, with few proprietary schools and only 
six state supported post secondary schools. 

Public Schools 

For the six state supported postsecondary schools in Idaho, the 
number of completers is collected five times per year, corresponding 
with scik "sessions." A follow-up of program completers is done once 
each year through a mail survey of all students completing their 
programs between July 1 of the previous year and June 30 of the current 
year. The survey is conducted in the fall of each year so that 
completers have been out of school for at least three months. 

The major components of the survey are whether the completer is 
employed, a self-report on whether the skills learned in vocational 
training are being used in their current position, and initial wages. 
Students who do not respond to the mail survey receive a follow-up 
telephone call . 

The surveys are conducted by the individual schools and must be 
submitted to the state Division of Vocational Education by January 1. 
The information is provided to the state by program within an 
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individual school. The state provides the forms for reporting the data 
on completers and the follow-up information to the schools. Data are 
reported by school and by program. 

The only check on the quality of the data occurs during on-site 
evaluations conducted by state staff every five years for accreditation 
renewal. The data reporting system is also informally checked during 
regular deans' meetings and contact between school staff and state 
staff. There has been no effort to use third party data, such as state 
wage data, to verify or supplement the information reported by the 
schools. 

There are plans to move toward collecting limited information from 
all enrol lees, whether they complete the program or not, on an annual 
basis. Additional information from a sample of students would be 
collected for special studies. A study of lifetime earnings has been 
discussed as a topic for one of the special studies. 

The State Board publishes a catalogue that outlines all higher 
education programs in the state. In addition to a narrative report 
prepared by the Division of Vocational Education, the number of 
enrol lees and number of completers are included for all programs, and 
placement rates for some programs are included. The report is provided 
to the schools and to the general public. Information on initial wages 
is reported to the schools, and they are free to use or disclose the 
information as they wish. 

Some career planning and placement offices in the schools make 
extensive use of this information in advising students. The state 
funds one of these offices in each postsecondary vocational school. 
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As the result of follow-up data collection, schools have closed 
out programs, reallocated money among programs, increased recruitment 
efforts, developed alternative delivery methods, and restructured the 
curriculum. 

Some schools add additional items to the survey or conduct their 
own data collection efforts. The schools monitor gains in basic 
skills, using an "Asset Test" developed by the state, but do not report 
the results to the state. Individual schools also collect information 
on success rates in career l icensing tests, loan default rates, and the 
reasons students drop out of the programs. None of this information is 
reported to the state. 

The state expects to be doing a study of postsecondary vocational 
education in the near future. It will be a modification of a study on 
secondary vocational education that was completed recently. 

Proprietary Schools 




There are currently few proprietary schools in the state, although 
some are beginning to move in from other states. At the current time, 
there is no follow-up study of program completers required and no data 
are collected from these schools at the state level. 
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Illinois 



Each August 1, community college systems ^ JiS Illinois submit data by race and sex on 
stu&nmwhow^ 

programs conduct an occupational follow-up study artel report on program-related 
placement, job and program satisfaction, and hourly salaries. Work has just begun on an 
employment tracking system using Employment Security files matched with student Social 
Security Numbers. Under new r^uiremcnts, proprktary schools mast submit student 
enrollment agreements that include program informaUon, cost, completion rates, and 
placement data-the latter only when used in advertising. Schools arc subject to probation 
and license revocation if their completion rate falls below 50 percent 



Organization of Post secondary Vocational Education 

The Illinois Community Colleges Board oversees 50 schools. Of the 
661,000 students enrolled in credit courses in these schools, 263,000 
are in vocational courses. The Nonpublic School Approval Section of 
the Department of Recognition and Supervision und«r the Board of 
Education oversees the 210 proprietary schools that have programs for 
which licensure 1s not needed for practicing the trade, with an 
estimated 50,000 students attending these schools. Programs leading to 
a state license are the responsibility of the relevant licensing 
boards. 

P ublic Schools 

The community colleges implemented a new evaluation system in 
1989-90. Under this unit record system, the colleges provide the state 
with substantial data on each student enrolled. Completion information 
is also entered into the system; on August 1, schools submit the data 
on students who had completed in the previous school year. A completer 
is one who has met all the requirements of an approved curriculum and 
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received a certificate or degree. Completion data are available by sex 
and racial/ethnic status. Completion rates are not yet used, but the 
state is considering developing a formula as the result of the Perkins 
requirements. 

Each year an occupational follow-up study is conducted on 20 
percent of the programs. The five-year schedule has been planned so 
that no college has a heavy load several years in a row. Items on the 
survey of completers include current educational status and present 
employment. Respondents must indicate if their jobs are related to 
their college program and / the jobs are not related to give the 
reason why. Other survey items are the number hours worked each week, 
present hourly salary before deductions, job satisfaction, satisfaction 
with the program of study, name of the employer, job title, and job 
duties. 

Occupational follow-up surveys are done by the individual colleges 
in March of the year following completion. All questions provided by 
the state must be incluaed but the colleges nay add questions. The 
survey may be done by mail or over the telephone. Colleges enter the 
information into a computer and send the data to the state. The state 
does the analysis of the data but the colleges might also analyze the 
data. 

Schools also must conduct an employer survey. They may do this by 
mail but the state recommends that they either conduct in-depth 
interviews of a sample of employers or assemble focus groups of 
employers for discussion sessions. Employer follow up is begun as soon 
as student information is received and must be submitted by August 1. 
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All schools comply in submitting data on individual students. 
Some individual data items are sometimes missing, most frequently the 
racial/ethnic status and the county code of residence. These items are 
missing because the students do not supply the information. According 
to one source, because the data are on a unit record basis, the schools 
are less likely to submit inaccurate information. 

To receive state funding, a college must be recognized by the 
Illinois Community Colleges Board. As a part of the recognition 
process, the colleges must do a self-study every three years and the 
state must make an on-site visit. One of the standards in this process 
is that the school must submit accurate data in a timely manner. 

Each college will receive a summary report on its follow-up data 
as well as a state summary. Uses of the data have yet to be 
determined. In the past it has been submitted to the Board and to the 
community colleges. On request, it has been provided to the 
legislature and the Governor's subcabinet. It is expected that 
information about individual programs would be provided by the local 
colleges. A statewide task force is looking into disclosure so that 
colleges will handle it consistently. 

In the past, data collection was done primarily to make management 
decisions. With the new disclosure requirements, about equal weight 
will be placed on consumer rights purposes and management decisions. 
The decision to discontinue or restructure a program is made by the 
Individual colleges. The state does not have the authority to do this, 
although the Board of Higher Education can make a recommendation to a 
local board of trustees that a program is not viable. 
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The Community Colleges Board h*s just begun to work with the 
Department of Employment Security on an employment tracking system. 
The social security numbers of completers are being checked against 
Employment Security files to determine if the individuals are 
1) registered with the job service, 2) drawing unemployment, 
3) employed, 4) place of employment, and 5) quarterly earnings. One 
problem with using these data is that they do not indicate occupation, 
just the location of employment. 

The Community Colleges System is considering having students sign 
a declaration of intent that includes the program in which they are 
enrolled. This would simplify the task of assigning the student to the 
correct program, which has been a problem in the past. Students also 
would indicate when they expect to complete, which would be useful in 
disclosing data to the public. For example, some students never had 
the intention of completing a full course of study; they just wanted to 
take two welding courses. How this type of student is counted could 
have a major effect on the calculation of a completion rate. 

Proprietary Schools 

In January 1989, Illinois revised its statute pertaining to 
proprietary vocational education. The rules are expected to be ready 
in April or May 1990. It is recognized that the statute needs some 
technical corrections. 

Under the new requirements, schools must submit each student's 
enrollment agreement to the state. Among the items that must be 
contained in each of these agreements are the name and description of 
the student's course of instruction, the total cost of the course of 
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Instruction and all other services, a buyer's right to cancel, and most 
of the disclosure information described below. 

Schools must provide the number of students who enrolled in the 
school during the school's last fiscal year and the number and 
percentage who did not complete the course of instruction. Completion 
rates also will be calculated by comparing the number of students 
enrolled in the year in which they expect to complete to the number who 
actually do finish. A completer is one who has graduated or completed 
a program. If a school's completion rate drops below 50 percent of the 
industry average within the state, the school will be put on probation 
for one year. If the completion rate is below 50 percent for a second 
year, the school's approval to operate in the state will be revoked. 

Under the current requirements, schools muct provide placement 
rates to the state only if they advertise that they have a placement 
and assistance program. Then, they must provide the number of 
completers who sought assistance from the placement service and the 
number who were placed. Any kind of placement counts. Under the new 
requirements schools will report placement rates for completers who 
used the placement assistance and for those who did not use the 
assistance. Initial wage rates will be reported if the schools have 
the data after making reasonable efforts to contact completers by mail. 

Schools will report their Jata in their renewal applications that 
are due April 1. These data, however, will be based on each individual 
school's fiscal year. The data will be monitored during the on-site 
audits that the state already makes at least twice a year to each 
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school. During these audits, faculty qualifications, facilities and 
equipment, and school financial condition are reviewed. 

Individual schools already are required to give the number of 
students enrolled 1n a program, the number completing, and the number 
placed to new students at the time of enrollment. It 1s reported that 
less than perfect data are being submitted currently. 
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In diana 



Currently, public schools in Indiana report student information on course enrollment, credit 
hours completed, and major; student retention and completion data could be computed on 
these data. A performance measurement system is under development Proprietary schools 
provide enrollment, dropout, and graduation information to the state. The state calculates 
completion rates and obtains placement rates through a survey of a sample of employers, 
the state also collects information for annual accreditation renewals. 



Organization of Postsecondarv Vocational Education 

The Commission on Higher Education oversees postsecondary 
vocational education in indiana. The Indiana Commission on Vocational 
and Technical Education has some overlapping oversight 
responsibilities. The majority of the programs are located at Indiana 
Vocational Technical College. Other programs are provided at a second 
two-year technical institution and at five of the four-year 
institutions. In addition, both secondary and postsecondary programs 
are provided in the state's 27 vocational technical training centers. 
The state does not have a community college system. 

The Commission for Postsecondary Proprietary Education oversees 
all proprietary schools except those regulated by other state agencies, 
such as cosmetology and barbering colleges and real estate institutes. 
Over 200 schools are regulated by the commission and they enroll about 
30,000 students. 

Public Schools 

In 1987, the General Assembly reorganized vocational and technical 
education. The Commission on Higher Education is in the process of 
developing a performance measurement system for these programs. 

64 

71 



Currently, the state has a student information system that 
contains data such as course enrollment, credit hours completed, and 
major. Schools submit this information on computer tape in a 
standardized format. Student retention and completion can be 
determined from these data. The Commission on Higher Education submits 
a list of programs with low enrollments and few graduates to the 
respective institutions to have them either justify the value of these 
programs or shut them down. Five occupational ly specific programs have 
been eliminated under this process. 

In 1987, the General Assembly also ordered the Commission on 
Higher Education to conduct an attrition-persistence study. The 
Commission's study tracked all students who entered in 1984-85 and 
declared from the beginning that they were seeking a degree. These 
students were followed over a five-year period. Data were obtained 
from the student information system. It is expected that similar 
groups of students will be tracked in the future. 

In summer 1989, Phi Delta Kappa conducted a study to formulate a 
list of criteria for evaluating vocational technical education 
programs. A state panel has been established to review and select 
appropriate criteria that would serve as useful measures in evaluating 
programs. No follow up of graduates is being done currently. 

Proprietary Schools 

To be accredited in Indiana, proprietary schools must be reviewed 
by the State in the areas of financial stability, admissions standards, 
enrollment forms or contracts, program content, instructor 
qualifications, student record keeping, and refund policies. 
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Institutions must be bonded and their recruiting agents licensed. 
During an on-site visit, the school's programs and physical facilities 
are evaluated. Proprietary schools provide enrollment, dropout, and 
graduation information to the state. Completion rates are calculated 
based on a comparison of the number of completers to the number who 
started the program. A completer is one who has finished a program. 
Both full-time and part-time students are included in the calculation; 
most students are enrolled full time. 

Placement rates also are collected by the state through a survey 
of a sample of employers. The employers are asked to assess whether 
the placement is related to the training. Separate rates are 
calculated for graduates who are working in their field of study and 
those who are not. Initial wage rates are not collected by the state. 
However, in the graduate records of students, the schools must retain: 
1) job placement given if job placement assistance is offered by the 
institution and 2) place of employment and job salary after graduation. 

Schools submit data annually in the accreditation renewal and a 
report to the state, but the Commission does not prepare a report on 
the information. The data are primarily used for management decisions. 
The state audits a random sample of 10 percent of the completion and 
placement rates for each school. 
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iowa 

< ^ a "vital signs* self-evaluation every five years that 

includes" such info placement rates; student perceptions of 

vocational training; employer satisfaction of students' technical knowledge, work attitude/ 
work quality, and comparison to others similarly employed; and per student costs. A criteria 
checklist collects information about programs, funding, staffing, facilities, etc. The state 
presently does not collect any outcome data from proprietary schools, but is considering 
some measures. 



Organization of Postsecondarv Vocational Education 

The Bureau of Area Colleges in the Department of Education 
oversees 14 community colleges and 1 vocational/technical institute, 
enrolling about 500,000 students. The Vocational Education Advisory 
Council oversees vocational education programs in the community 
colleges and the 435 secondary school districts. About 375 of the 
secondary districts offer vocational education at the postsecondary 
level. Altogether, the Vocational Education Advisory Council oversees 
about 8,000 postsecondary vocational education students. Two 
proprietary business schools are members of the Postsecondary 
Coordinating Council and also of the Iowa Association of Independent 
Colleges. 

Recent legislation has established strict vocational education 
standards that become effective July 1, 1992. All postsecondary 
education will use competency based instruction, articulation with 
postsecondary instruction, and minimum competencies in six areas- 
leadership skills; entrepreneurial skills; job seeking, job keeping, 
and adaptability skills; knowledge of emerging technology; 
strengthening of basic skills; and employment skills relating to 
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industry training standards. The Board of Education was charged with 
the responsibility of establishing minimum competency levels in all 
these areas. About 25 committees will be set up to investigate 
attainable minimum competency levels that also will become 
accreditation standards for local schools and community colleges. 
Proprietary schools will be exempted from the vocational education 
standards but it is possible that the system will be incorporated into 
the approval process of these schools. 

Pending legislation proposes the establishment of a 14-member 
higher education strategic planning council with statutory powers over 
all higher education. 

The Iowa College Aid Commission is the official loan guarantee 
agency for the state. The Commission requires schools with high loan 
default rates to provide placement rates and initial salaries. Schools 
must also identify steps taken in minimize loan losses. 

Public Schools 

Currently, the official student outcomes program for community 
colleges in Iowa has two components: a "vital signs" analysis and a 
criteria checklist. The vital signs analysis is a self-evaluation 
system in which schools must provide data by program on factors such as 
completion rates, placement rates, student perception of vocational 
training, employer satisfaction, and per student cost. Earnings data 
are provided on a voluntary basis. The individual schools determine 
how to calculate completion rates. Placement rates are defined as the 
comparison between the number of completers employed and the number 
available for work. Schools survey employers for their satisfaction 
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with the job performance of graduates 1n the areas of technical 
knowledge, work attitude, and work quality. They also are asked to 
compare the graduates to others who are employed in similar positions. 

For the criteria checklist, schools must provide documentation on 
program offerings, program evaluation, instructor load, funding, 
staffing, and facilities and equipment. 

School' submit the information for the self -analyses to the state 
every five years, but some schools collect the information annually for 
internal use. Teams of people from the state then visit the schools 
and prepare a report on each school. Schools must submit progress 
reports for the subseqi ~nt years. 

Proprietary Schools 

The Postsecondary Coordinating Council does not require member 
institutions to submit any outcomes assessments. Data on completion 
and placement are reported to the Federal government as part of the 
annual financial aid audit, but state agencies take no role in this. 
The state is considering requiring the proprietary schools to submit 
graduation and placement rates. 
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Kansas 



Public postsecondary vocational schools in Kansas report information on number of 
completers and placement rate by gender and race, with a required 70 percent placement 
rate for students within two years after graduation. Social Security Numbers are used to 
track students whose placement status Is "unknown/ Average annual earning, job 
satisfaction, success rates on licensing tests, basic skills test scores, and employer ratings of 
employee job skills are reported/ No performance measures are collected from proprietary 



Organiza tion of Postsecondarv Vocational Education 

The Community and Vocational Education Division of the Department 
of Education oversees 14 approved vocational/technical schools and 19 
community colleges. The vocational/technical schools enroll about 
42,000 students; approximately 100,000 students attend the community 
colleges and business/industry short-term programs, such as seminars 
and one-month courses. The division also licenses proprietary schools 
on an annual basis. The state has about 73 proprietary schools; with 
the rapid turnover among these schools, the total count changes 
monthly. 

Public Schools 

The state collects information on the number of completers in each 
instructional program. A completer is defined as one who has been 
awarded a degree, diploma, or certificate or has finished all courses 
in an approved program. Completion information is available by gender 
and racial/ethnic categories. 

The number of completers placed and the placement rate of 
completers are collected by the schools through a survey and reported 
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to the state. Six categories of placements are used 1n the survey: 
unrelated placement, related placement, military service, continuing 
education, unemployed, and **atus unknown. The last two categories, 
unemployed and status unknown, are excluded from the calculation of 
placement rate. Schools are required to have a 70 percent placement 
rate during the first two years after completion. Schools must explain 
low rates and may lose approval for programs with low placement rates. 
Follow up on students is done three months after initial placement. A 
second follow up is done three months after that, i.e., six months 
after initial placement. 

Schools submit the social security numbers (SSN) of program 
completers to the state Education Department. When a student's 
placement status is reported as "status unknown," the SSNs are cross- 
referenced with state wage. Completers who are found not to be 
employed receive state placement help. 

The schools conduct a survey of completers one year after leaving 
the schools. A survey of employers also is conducted. The schools 
then report average annual earnings, job satisfaction, success rates on 
career licensing tests, and basic skills test scores to the State 
Education Department. Employer ratings of employees' job skills are 
also assessed. 

Schools are required to submit enrollment and placement data at 
the end of the calendar year. By February 15 of the following year, 
schools must submit a placement report. The Department compiles this 
information into two reports: the Annual Voca tional Education 
Accountability Report , which goes to the legislature, and a report on 
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placement rates and average salaries. The Information is used for botn 
management decision-making and consumer rights purposes. Press 
announcements frequently are made. 

Schools are site visited every year. Every five years, the state 
conducts a product audit that involves both a financial audit and a 
visit by an accreditation team. Areas funded with public funds are 
expected to show program Improvement. 

Gai^s in basic skills also are assessed for each program. Tests 
are administered at program entry and completion. Students are 
assigned to remedial programs when appropriate. 

Proprietary Schools 

The state does not collect any performance measures from the 
proprietary schools. Only the number of enrol lees by program are 
submitted. Instead, the state uses the standards established by the 
pertinent accreditation agency. For real estate schools that are 
unaccredited, the agency sends the curriculum to the Kansas Real Estate 
Commission for evaluation of performance standards. 

The Department tried to compile data on all currently enrolled 
students 1n proprietary schools. The high turnover made the task 
Impossible. Only a few schools responded to the questionnaire over a 
six-month period. 

The legislature has mandated that an occupational handbook be 
prepared using completion and placement information from postsecondary 
schools. Proprietary schools have been invited to participate. 




72 



79 



Kentucky 

Vbc^ (ftj* finance 

facilities, enrollment trends), completion rates, placement one year after teaytog school 
(through survey results), and, it placed, employer survey data, there Is no systematic 
collection of performance data i)y the state from either community colleges or pi^t^tary 
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Organization of Postsecondarv Vocational Education 

Kentucky has 22 vocational/technical schools; 5 are health related 
and the remaining 17 are state vocational/technical schools. Some 
secondary students attend 14 of the state vocational/technical schools. 
Altogether the vocational/technical schools enroll about il,000 
students. The Higher Education Council oversees 21 community colleges, 
7 of which are on the campuses of four-year institutions. About 36,000 
students attend the community colleges. The Education and the 
Humanities Cabinet oversees 92 proprietary schools, which enroll about 
28,000 students. 

Public Schools 

Completion rates are collected from the vocational/technical 
schools. A completer is a person who has the competencies to complete 
a predetermined occupational code title from the Dictionary of 
Occupational Title Codes (DOT). Each program may have several exit 
points below the diploma level. 

Placement information is collected in a student survey conducted 
by the school instructors one year after the student has left the 
school. Placement categories include employment in jobs related to 
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training, employment in jobs unrelated to training, continuing 
education, and not available for work. The relationship between 
placement and training is determined by the teacher conducting the 
survey. At one time, the student survey contained questions about 
earnings, but these have been deleted. 

For students who are employed in jobs related to their training, 
an employer survey is done by the schools one year after the student is 
on the job. Earnings information was asked in 1989, but many employers 
would not complete the survey because they did not want wage 
information known. In 1990, the survey of employers asked about 
average wage data in different occupational areas. This also produced 
a low response rate and is not expected to be asked again. 

Success rates on career licensing tests are reported to the state. 
Students must pass an exit test of Adult Basic Education in the areas 
of language arts and computation. However, scores on this test are not 
sent to the state. 

Schools report the data quarterly on a computerized, on-line 
system. Each year, one-third of the schools participate in a formal, 
program-by-program evaluation and validation. 

State wage data are used to develop labor force projections for 
program development. The difficulty with these data is that they show 
a statewide picture while the state prefers to look at local markets. 
The state also has information on school financial condition, 
facilities and equipment, and student enrollment trends. 

The data are used to ,«,<e programmatic decisions, such as 
modifying or phasing out programs. The Governor and legislature are 
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provided Information on request. The postsecondary schools receive 
information on the program evaluation and modifications. No effort 1s 
made to provide Information to students and their families. 

At present, no systematic collection of assessment Information 
exists for the community colleges, but the Higher Education Council 1s 
pushing for this type of requirement. The legislature 1s considering 
funding for assessment. 

Proprietary Schools 

The proprietary schools in Kentucky currently operate under 
minimal licensing standards; no measures of performance are collected 
by the state. Legislation has been introduced that would require 
placement and graduation rate data to be disclosed to all students 
before they are enrolled. 
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Louisiana 



Louisiana currently collects completion and placement data from each program in the 
vocational/technical schools. A ratio of completers to leaver* must be provided* Placement 
data are collected through a suivey of completers that also collects information on 
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relatednessof jobto 
students* technical knowledge, work attitudes and abilities, and 
training. |to peil^a 




Organization of Post secondary Vocational Education 

Louisiana does not have community colleges. The Office of 
Vocational Education in the Department of Education is responsible for 
45 postsecondary vocational/technical schools. The Division of 
Proprietary Schools in the Department of Education oversees the 225 
proprietary schools with programs in areas that do not require a 
license. Schools for cosmetology, bartering, real estate, and other 
fields requiring a license are under their respective boards. 

Public Schools 

Louisiana Is in the process of finalizing a Vocational Technical 
Information Network that will ultimately automate postsecondary 
vocational education follow-up activities. 

Currently, completion and placement data are collected from each 
program in the vocational/technical schools. A completer is defined as 
any student who finishes a planned sequence of courses or activities 
designed to meet a vocational occupational objective that teaches 
entry-level job skills. The individual must meet all the requirements 
of the school for program completion whether or not tne person 
graduates. A ratio of completers to leavers must be provided. 
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Placement data are collected through a survey of completers conducted 
by the school. The survey 1s used to determine the number of 
completers who are employed In jobs related to training, employed In 
jobs not related to training, employed full time In the military, 
continuing their education, not employed but looking for work, and not 
attending school. Employer follow-up data also are collected by the 
State. Employers of those who are working In a field related to their 
training are asked If the Individuals had technical knowledge related 
to the job, a desirable work attitude, an ability to produce quality 
work, and more than adequate overall vocational training. Employers 
also are asked to compare the vocational training of both completers 
and leavers to employees who did not participate In a vocational 
program. 

Summaries of the follow-up reports from the schools are published 
In an Annual Executive Summary. The State 1s In the process of 
establishing an automated Vocational Technical Information Network that 
also will automate the completer/leaver/ employer follow-up activities. 

Proprietary Schools 

No performance measures are collected from the proprietary schools 
and no reports are generated. The state does regulate advertising from 
these schools, requiring substantiation of claims made for placement 
and earnings. 
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Main? 



Maine collects completion and placement data from its six technical colleges, although 
common definitions are not used. A graduate mail survey collet * employment satisfaction 
and initial wage rates, the state also collects some school char, cteristic data, Proprietary 
schools submit the data needed to complete IPEDS forms. 



Organization of Postsecondary Vocational Education 

The Maine Technical College System consists of 6 two-year 
technical colleges that enroll 3,000 full-time students and 6,000 part- 
time students. Maine has 4 proprietary postsecondary schools and 13 
nonprofit private schools. The state has no direct oversight over 
private schools. 

Public Schools 

Completion rates based on the number of students starting compared 
to the number of completers are collected. Completers are those 
students who have satisfactorily completed a program, certificate, or 
degree. The calculation procedure is not standardized across the 
state. 

Placement information also is collected by the State, but each 
school uses its own definition of the data categories. Data are 
collected on the number of students employed in jobs related to their 
education, the number in unrelated jobs, and the number continuing on 
to four-year schools. Employment satisfaction and initial wage rates 
also are collected. This information is obtained through a mail survey 
of all graduates; those who do not respond are contacted by telephone. 
In some years, a survey is made of all employers mentioned on the 
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graduate follow ups. Some schools conduct five-year follow-up surveys 
of their graduates. Career licensing tests success rates are collected 
when applicable. 

The completion and graduate follow-up data are collected annually. 
They are Included in a report that 1s given to the Board of Trustees, 
the Governor, and the legislature. All academic and management 
divisions of the schools receive the information and conduct their own 
analyses. Students are informed of placement rates. The Information 
is used primarily for management decisions. Program evaluations are 
based on placements. 

The state also collects information on school financial condition, 
student enrollment trends, and facilities and equipment. 

Proprietary Schools 

The private schools return Integrated Postsecondary Education Data 
System (IPEDS) forms to the state, which makes copies of them. From 
these data, the state compiles summary statistics for Its own use. The 
state does not collect any additional data from the private schools. 
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Maryland 



All community colleges in Marytind annually report emollments, number of completers, tod 
number of plw by sex and Blwrnaal <oll< studies of complet< used 
to gather uuoifmi^ 

ant relationship of placement to training. An employee satlsfoctioit survey obtains 
information on training program quality and relationship of placement to training. Plans 
are in process to use unemploymetit insurance wage data as a performance outcome 
measure. Proprietary schools sut/mit information on enrottees, dropouts, completers, and 
placements. Completion and placement rates ate reported by gender, race, and program, 



Organization of Postsecondarv Vocational Education 

Maryland's postsecondary education schools are overseen by the 
state Higher Education Commission. Vocational programs fall under two 
categories: the collegiate (public) and noncollegiate (nonpublic 
vocational/technical Institutes). The collegiate programs are 
contained in 19 community colleges and the noncollegiate programs 
operate in 175 proprietary schools. About 100,000 students are 
enrolled in the community colleges and about 60,000 students attend the 
proprietary vocational schools. 

Public Schools 

All corcminity colleges report performance outcomes including 
completion rates, placement rates, and earnings information annually at 
the request of the Higher Education Commission and the Board of 
Community Colleges. Success rates in career licensing tests are also 
collected from some of the licensing agencies. 

The schools also report on initial part-time and full-time 
enrollment, number of completers, and number of Individuals placed by 
sex and race to the Higher Education Commission. Completion rates are 
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calculated for both full-time and part-time students, taking into 
account those who dropped out of programs or are on vacation or leave 
of absence. However, the Commission has found that most community 
college students are not seeking the associate degree so completion 
rates are mostly insignificant. Placement rates, on the other hand, 
receive serious attention and determine survival or Initiation of a 
program. Only those placed in training-related jobs are included in 
the calculation. Several programs in community colleges have been 
terminated for low placement. No site visits are made to participating 
schools. The information collected is disseminated in the Commission's 
Annual Report, which is available to participating schools, 
policymakers, and the general public. 

The State Board of Community Colleges conducts biannual follow-up 
studies of completers to determine their education outcomes four years 
after graduation. During this time, the Board compiles information on 
completion and placement rates, relationship of placement to training, 
success rates 1n career licensing tests, and current annual earnings. 
In between the biannual follow-up studies, the Board has conducted 
mini -surveys of nonreturning students, and attempted to follow up on 
all entrants. Data generated are used to advise the community 
colleges. 

Completion rate, computed on a program basis, is the ratio of the 
total number of completers to the program's first time/full-time 
enrol lees at four years prior to current follow-up study. Completers 
must be recipients of the associate degree or certificate. Enrollment 
data are collected from the colleges every fall semester and completion 
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data at the end of the fiscal year. Computed completion rates are 
calculated for each community college. The Board uses completion rates 
for program evaluation. 

To measure placement rates, survey questionnaires are mailed to 
all graduates one year after completion by the schools. The survey 
Instruments are developed by the Commission on Higher Education, and 
the data are summarized and transmitted to the Commission on a 
standardized form. Respondents are asked, among other things, whether 
they are employed full-time or part-time, current annual salaries, and 
whether or not their jobs are related to training. A graduate must be 
a full-time employee in a training-related capacity to be considered 
placed. Therefore, the placement rate is the percentage ratio of the 
total number placed in full-time, training-related jobs to the total 
number of respondents for that survey period. Separate rates are shown 
by the programs within a college. The employer satisfaction survey, 
conducted only with student's consent, is used to determine 
relationship of placement to training. Employers are asked on an 
evaluation questionnaire to rate program quality based on the 
completer's job performance. Gains in basic skills are not collected 
at the state level although individual colleges administer their own 
competency-based tests at entrance and exit. The Board, however, 
collects first time passing rates in some career licensing exams such 
as nursing and allied health. Plans are underway to give the Board 
access to the state wage data for use in the analyses of performance 
outcomes. 
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The school -provided data are used to identify weak programs 
through the Board's annual evaluation system. After each follow-up 
study, the Board compiles a program profile for each program offered in 
the colleges. Three weak programs per college are analyzed in detail; 
they are targeted on the basis declining enrollment, sharp cost 
increases per credit hour, and completer dissatisfaction. Faculty is 
then asked to do a self-study of the identified weak programs. 

Plans are underway for colleges to begin using performance 
indicators. Colleges will be required to submit an assessment 
procedure, provide mission statements, and establish accountability 
programs . 

The Board makes annual site visits to conduct a product audit as 
part of the institute nal approval process. The team primarily 
verifies the follow-up survey data, for instance, the date the student 
entered the program, whether or not student completed necessary 
courses, and the program listed as the student's major. Independent 
CPA auditors verify enrollment data. On the whole the Board has good 
working relations with the colleges. The data collection efforts have 
resulted in some major Board decisions as well as policy initiatives. 
Included among them were: 

o Discontinuation of 104 programs in the last 12 years due to 
enrollment decline or low placement. Colleges seeking to 
reinstate discontinued programs must demonstrate proof that 
employment exists for all potential graduates, e.g., mental 
health assistants. 

o A follow-up study of nursing graduates during the recent 
nurse shortage that showed that the majority of those trained 
in crinmunity colleges were employed in full-time nursing. 
This resulted in the legislature authorizing additional 
funding for nursing prc^jns in the community colleges. 
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o A follow-up study of the continuing education population that 
found that most were working in areas related to the course 
work they were taking. This outcome also generated increased 
funding for community college programs. 

Information gathered is widely disseminated through numerous 

reports distributed to participating Institutions, boards of trustees, 

the Governor and the legislature, and the general public. The reports 

include: 

o Annual Report of programs identified as weak and needing a 
self- study. 

o An Annual Performance Profile Report on the colleges, sent to 
their local boards of trustees. 

o Methodological reports explaining each survey; the colleges 
have used follow-up survey reports for recruiting purposes. 

o Informal analysis reports, sent back to the colleges with 
encouragement that the information be shared with local 
communities and boards of trustees. 

o Other specialized reports and analyses compiled for various 
state agencies and published bulletins. 

Proprietary Schools 

Annually, proprietary schools submit data by program on the total 
number of enrol lees, dropouts, those on vacation or leave of absence, 
completers, and placements. A completer is defined as one awarded a 
diploma or certificate. Completion rate, expressed as the percentage 
ratio of total completers to the total initially enrolled, is 
calculated separately for full -time and part-time students 1n each 
program, and Is broken down by gender and race. Placement must be 
training-related to be included In the calculation of placement rates. 
Placement rate, expressed as the ratio of those placed in training- 
related jobs to the number of completers, is calculated by program 
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within a school, and 1s broken down by gender and race. The schools 
determine the relationship of placement to training. Data are 
collected on a standardized form designed by the Higher Education 
Commission. 

Each school receives an annual site visit during which reported 
data are audited and overall Institutional operation reviewed. Labor 
market conditions play a major role in curricula development. Several 
programs have been terminated or initiated according to prevailing or 
projected labor market needs. 

The information collected 1s disseminated in the Commission's 
Annual Report, distributed widely to participating schools and the 
general public. 

Students in proprietary schools are eligible for some of the state 
financial aid. In addition, students attending nationally accredited 
proprietary schools may receive legislated scholarships, and are also 
counted among the state's 32,507 current recipients of student loans 
under the Stafford (and other) loan programs. The Maryland Higher 
Education Corporation serves as the official student loan guarantee 
agency. The state's current default rate is around 10 percent, a 
quarter of which is attributed to proprietary students. 
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Massachusetts 



:\,Thc state collects basic sldUs data and number of degrees conferred for community college 
students in Massachusetts. It recently conducted a survey to determine what measures of ; 
effectiveness ate being used. Every two years, proprietary schools submit a licensing 
application form that includes number of graduates and placements. State law prohibits the 
collection of earnings information. 



Organization of Postsecondarv Vocational Education 

In Massachusetts, the Board of Regents of Highe M Education 
collects information on the 15 community colleges, which enroll 
approximately 73,000 students. The Department of Education licenses 
the state's 140 proprietary schools and oversees regional vocational 
schools at the secondary level. 

Public Schools 

The state collects Information on the number of degrees conferred 
rather than completion rates. The community colleges submit the data 
in written form on an annual basis, but they are not required to 
provide it. The state uses the information primarily for management 
decisions; the data are not included in a report. 

Most of the community colleges collect some follow-up data on 
their students. This information 1s not se.nt to the state. IPEDS data 
are the only source of placement data used at the state level. 

Massachusetts will begin to collect gains in basic skills at the 
state level in September 1990; the details are still being worked out. 
The state is also starting to collect data on school enrollment trends 
and school financial condition. 
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The state recently conducted a survey of the community colleges to 
determine what measures of effectiveness are being used. The survey 
contained items on measures such as degrees conferred and follow-up 
Information. The data have not been compiled. The information will be 
used internally and not released to the public. 

Proprietary Schools 

Every two years, proprietary schools must submit a licensing 
application form to the state. Some performance measures are included 
in the form. The number of graduates is collected; both full-time and 
part-time students are included in the count. The number of students 
placed is also collected. 

The State Department of Education is prohibited by state law from 
collecting earnings information. 

The applications are evaluated for licensing purposes only; no 
other use is made of the information. The state is currently reviewing 
the legislation in this area. It expects to be looking at completion 
and placement issues. 




Michigan 



Michigan collects information on enrollments and number of completers from its community 
colleges. There has been resistance to collection of additional performance data. Bach 
spring, the state collects enrollment and graduation data from the proprietary schools. 



Organization of Postsecondarv Vocational Education 

Michigan has 29 public junior and community colleges, which are 
attended by 224,000 full-time and part-time students. About 110,000 
students attend the state's 55 vocational centers but only about 4,000 
to 5,000 of the students are adults. The Higher Education Management 
Service Area oversees 310 proprietary schools. 

Public Schools 

The Department of Education collects information on enrollment and 
the number of completers from the community colleges. A completer is 
someone who has completed a certificate or degree; both part-time and 
full-time students are Included In the count. Total enrollments for 
each school are subdivided by gender and racial/ethnic status. It Is 
up to the Individual schools to decide If they will collect placement 
rates. The colleges collect some initial earnings Information but the 
rate of return of the Information 1s low. 

The Department of tducation has tried to move toward the Florida 
approach for collecting performance data but It was unable to get an 
agreement from the community colleges. No legislation exists to 
require this approach and the colleges successfully lobbied legislators 
to prevent such a system from being mandated. 
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Colleges submit the enrollment and completion data to the state on 
computer disks and the information is uploaded into the state's on-line 
database. A separate unit, the Community Colleges Services Unit, 
handles compliance and accuracy of the information received from the 
schools. 

An annual report is sent to the Governor and the legislature. 
Special topic reports also are produced. Postsecondary schools have 
access to the on-line data. The data are used primarily for management 
decisions, which tend to focus on fiscal matters. 

Information is given to students and their families on request. 
Most requests concern tuition rates. The Michigan State Placement 
Institute provides aggregated information by program, not by 
institution; the emphases of this information are career choices and 
cost-benefit analysis. 

Although programs at the vocational centers are primarily for 
secondary students, a small number of adults also participate. The 
adult population is composed of students completing their general 
equivalency diplomas (GEDs) and high school graduates taking 
occupational ly specific programs. Participation data are collected for 
these programs but the information is not reported separately for 
adults and secondary students. 

Proprietary Schools 

Every spring the state conducts a program inventory of proprietary 
schools. The state sends the schools a computer form that the schools 
must update. The schools must indicate enrollment and the number of 
graduates from the prior school year. In the past, schools have 
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reported student numbers that differ from the data submitted to IPEDS. 
It is thought that some schools are not counting students correctly. 
Consequently, the state is putting more attention on accurate student 
counts this year. Schools are requested to provide completion 
Information by racial/ethnic status but some do not. 

Some schools have placement information but they are not required 
to provide it to the state. Accredited schools are often required to 
conduct follow ups of their graduates as a part of the accreditation 
process. The legislature has been working on a bill that may require 
placement and earnings information. 

The state analyzes the enrollment and completion data but does not 
produce a report on 1t. The Board of Education is the only user of the 
data. To date, no major changes in policies or programs ha/e occurred 
as a result of the data. To remove the license from a school would 
take something dramatic; an enrollment drop would not provide just 
cause. 

During on-site reviews, the completion data are examined. Other 
factors that are reviewed are student complaints and facilities and 
equipment. The state must approve all faculty before they are hired. 
School financial condition is reviewed before a school receives Its 
license. 
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Minnesota 



A uniform tracking system for higher education, to be used in part to provide consumer 
information^^ mandated by the legislature to tor both public and proprietary 
schools. However, issuance of new Federal regulations has postponed development of the 
system. From the technical colleges, the state currently collects information on the number 
of completers and placement information, including relationship of placement to training, 
with each college determining its own method for contacting graduates. Community colleges 
provide completion data by sex and race, and collect placement data through graduate 
surveys. The Community College System is developing an automated information system 
and plans to collect wage rate data. A student outcome project was pilot tested at two 
community colleges. Proprietary schools report maximum and current enrollments and, if 
they have a placement service, the number of completers and placement rates. State law 
prohibits the collection of wage data from these schools. 



Organization of Postsecondarv Vocational Education 

The State Board of Vocational Education oversees 30 public 
technical colleges, which are attended by about 50,000 students and 
500,000 extension students. The 18 schools in the Minnesota Community 
Colleges System are attended by about 54,000 students of which 18 
percent are enrolled in vocational education programs. Private 
vocational schools are overseen by the Department of Education. 
Currently, Minnesota licenses 80 proprietary schools that do not grant 
degrees; additional schools are awaiting licensure. Approximately 
20,000 students attend the private vocational schools. 

Negative reports about postsecondary outcomes appeared in the 
local press in Minnesota during the 1980s. Consequently, in 1989, the 
legislature mandated the state's public postsecondary governing boards 
and private proprietary institutions to review and/or establish 
procedures for collecting data on the job placement of the graduates of 
their vocational programs. The Higher Education Coordinating Board was 
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required to coordinate the reviews and submit a report to the 
legislature by February 15, 1990. The information from this report was 
+o be used to develop a uniform tracking system for higher education. 
This system would be used in part to provide consumer information to 
students and prospective students. 

While the state was revising its placement tracking systems, the 
federal government issued regulations on the reporting of completion 
and placement daf for vocational programs below the bachelor's level. 
Since the federal reporting requirements differ from the state systems, 
it was thought that users would be confused by having to provide two 
different sets of data. Therefore, the report recommended the 
postponement of the development of the state's uniform tracking system. 
It also endorsed the efforts of the postsecondary community in the 
state to revise federal reporting requirements. 

Public Schools 

Technical Colleges. The state collects information on the number 
of completers from the public technical colleges. A completer is 
defined as one who has completed the graduation requirements of the 
college attended. The information on completers is categorized by rase 
and sex. 

The public technical colleges provide placement information to the 
state by January 15 each year. Each college determines how the 
graduates will be contacted--by mall, by telephone, or in person. In 
reporting the data to the state, the colleges may submit a 9-track 
magnetic tape, enter data on-line to the state computers, use their own 
software to produce and submit an ASCII file, or use manual reporting. 
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For each graduate from the previous fiscal year, the colleges must give 
the job title of those employed, a job title qualifier if necessary, 
and the reason that unavailable graduates are not looking for work. In 
determining the relationship between placement and training, the state 
uses a job title qualifier and has state curriculum specialists and 
other experts determine if the job uses at least 50 percent of the 
curriculum. The vocational technical system is required to close 
programs with related placement rates falling below a minimum percent, 
which for many years was 51 percent. However, one recommendation in 
the report on placement tracking systems was to remove reference to 
placement rates as a mandatory criterion for program evaluation. 

The public technical colleges collect wage data but this is not a 
state reporting requirement. Minnesota requests, but does not require, 
colleges to collect data on success rates in career licensing. Basic 
skills test scores also are reported to the state. These data are 
collected by an independent assessment firm. A booklet containing the 
analyzed data is sent to the colleges. Data are available to students 
and the- general public. 

Two processes are used to monitor accuracy of data from public 
technical colleges: product audit and fiscal audit. A product audit 
is used to assess placement relatedness. To check the accuracy of the 
data from each school, an auditor contacts a random selection of 
graduates as well as all graduates in certain selected programs. All 
colleges undergo a product audit annually. Fiscal audits are conducted 
on six colleges per year; during the site visits, student data are 
reviewed. 
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Pending legislation Is addressing the topic of consumer rights 
Information. It Is expocted that a task force will be established. 
Currently a moratorium prohibits technical colleges from publishing 
placement data. An effort Is underway to redesign the employment 
follow-up system, after which reporting will resume. 

The State Board of Vocational Technical Education is exploring the 
feasibility of working with the job service of the Department of Jobs 
and Training for obtaining job placement and employment follow-up 
information. 

Community Colleges. The community colleges provide completion 
information to the state. A completer is one who has been awarded an 
associate degree, diploma, or certificate; Information Is available by 
gender and racial/ethnic status. The agency Is currently developing an 
analysis system for completion rates. All colleges are required to 
collect placement information by surveying their graduates. A measure 
of placement has been formulated but not yet implemented. 

The Community Colleges System is developing new policies about the 
measurement of outcomes. Schools will be required to collect wage rate 
information. They also will be collecting success rates on career 
licensing tests annually from the private Institutes that do the 
license testing. Effective fall 1990, a mandatory assessment program 
will be Implemented to measure basic skills at the completion of a 
program; the agency is still seeking an appropriate instrument. 

The Community Colleges System has a centralized automated 
information system. The schools enter data such as enrollment, student 
demographics, completion, and placement directly Into the system. 
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Analysis systems for these data currently are being developed. No 
formal audits of these data are done. Printouts are sent to the 
schools two times each quarter to check on the accuracy of the data. 
Placement data are checked as a part of the regular biannual college 
review. A summary report of placement information is planned. 

A student outcomes pilot project was conducted at two community 
colleges. Among the components of the project were a placement survey 
of graduates one year after graduation, an extended survey of graduates 
five years after graduation, and a survey of non-returning non- 
graduates' placement, college evaluations, and college experiences. 

Proprietary Schools 

Schools must report maximum enrollment and current enrollment to 
the state. Overenrollment could result in licensure revocation but 
underenrollment is not considered a problem. Schools providing a 
placement service must report the number of completers and placement 
rates. Placement rate is based on the percentage of placed graduates 
who are 1n a job related to their training. The measurement is made 12 
months after completion. State law prohibits the agency from requiring 
schools to submit wage information. The agency contacts employers to 
verify placement rate data and to assess whether the placement is 
related to the training received; this is done at least annually during 
the licensing process. Placement tracking information, also collected 
by the Minnesota Association of Private Post-Secondary Schools, is 
included in the state reports. 

The information on proprietary schools is used by the legislature 
and the Governor to amend legislation and policies. Schools found with 
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deficiencies receive feedback about the conditions they must meet In 
order to retain their licenses. The state publishes a directory of 
licensed schools that Is available to the general public. 




Mississipp i 



Completion and placement rates are reported tor community college students In Mississippi, 
separated ov.t by sex, race, handicap, and disadvantagement Programs are ranked using z- 
scores, with the lowest 25 percent evaluated* For license renewals, proprietary schools 
submit information about completion and placement rates. 



Organization of Postsecondary Vocational Education 

The Junior College Division has approval authority over the 
vocational/technical courses at the 15 community colleges in 
Mississippi. However, all reporting is done through the Vocational and 
Technical Education Bureau in the Department of Education. About 120 
licensed proprietary schools enrolling approximately 20,000 students 
are overseen by the Proprietary School and College Registration 
Division of the State Department of Education. 

Public Schools 

The state collects completion rates for community college 
students. Completers are full-time students who have finished all 
requirements of a program. Data on progress toward completion also are 
collected. Placement rates are calculated for completers who are "not 
back in a program;" the individual schools decide if a placement 1s 
directly related, indirectly related, or not related at all to the 
training program. Separate completion and placement rates can '.2 
calculated by gender and race and for handicapped and disadvantaged 
students. 

The community colleges submit computer printouts on completion and 
placement rates annually. Starting in 1989, the state ranked programs 
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using 7-scores based on enrollment, retention, and placement. Programs 
1n the lowest 25 percent will be evaluated. This Information 1s 
distributed to the schools; teachers may review it to see how their 
programs are doing. It 1s not disclosed to students or the general 
public. It 1s used for management decisions only. It 1s expected that 
the ranking system will result 1n program modification or cancellation. 

The state sends a sample survey to the schools that can be used 
for student follow up. Schools are not required to use the form. When 
the forms are utilized, students return them to the schools; the data 
are used by the schools and not forwarded to the state. 

The state does collect information on school financial condition 
and facilities and equipment. 

Proprietary Schools 

The state receives some performance data from the proprietary 
schools every two years when the schools renew their licenses. 
Completion rates are collected and are based on the number of graduates 
in a program; both full-time and part-time students are included in the 
calculation. Placement rates based on the number of graduates in jobs 
related to their programs also are collected. The schools decide if a 
job 1s related to a program and use the Dictionary of Occupational 
Titles to make their decision. 

Some schools send computer printouts, but all must use a written 
format. The state uses the data for licensing renewal only. The 
schools use the data for program changes. Disclosure is not made to 
students or the general public. 
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The data are reviewed during site visits made every two years as 
a part of the licensing process. Since the state has no standardized, 
required record keeping procedures, problems arise concerning how the 
data are collected and recorded. 



99 

106 



Missouri 



A statewide performance indicator system recently has been established in Missouri for both 
public and proprietary schools. The system includes graduation rates, retention rates, 
academic and cognitive measures, student opinions, and license testing results. 



Organization of Postsecondarv Vocational Education 

The Coordinating Board for Higher Education has budget making 
responsibility over the 13 community college districts, each with three 
campuses. Altogether about 50,000 students attend the community 
colleges. The board has licensure responsibility over the 162 
proprietary schools, which enroll about 31,000 students. 

A statewide performance indicator system recently has been 
established. This will include graduation rates, retention rates, 
cognitive measures, a survey of academic performance, student opinions, 
a 4 a survey of withdrawing students. Vocational schools also will 
provide the results of licensure tests. The first year in which 
schools will be required io report is 1991. 

The new performance indicators system grew out of a desire by the 
Education Coordinating Board, the Higher Education Division, and the 
Governor to have a better sense of postsecondary education outcomes. 
The intent of the system is to provide feedback to the institutions 
about the outcomes and to foster improvement. Currently under 
consideration is the strengthening of the role of the Education 
Coordinating Board in the implementation of the indicators system. For 
example, the Board may be given the power to defund programs that fail 
to meet required standards. 
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Public Schools 

The Coordinating Board for Higher Education collects Information 
from the public schools on the make-up of Its student body. Among the 
data collected are age, sex, race, and degree status. As part of a 
Student Achievement Study, additional Information 1s being collected 
Including American College Testing (ACT) scores, educational degree 
aspiration, and vocational choice. 

Proprietary Schools 

The Coordinating Board for Higher Education is in its fourth year 
of jurisdiction of the proprietary schools in Missouri. As part of the 
requirements for certification, the Board collects statistical 
Information and publishes it in a Directory and Statistical Summary: 
Proprietary Sector of Missouri Postsecondarv Education . Among the 
Information Included are the prior education level of entering students 
and enrollment by gender, race, and age. 
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Montana 



During a period of transition in Montana, a set of common program evaluation standards 
is befeg developed for public vocational/technical centers. A manual for standardizing data 
collection and reporting is undergoing revision. For proprietary schools, the state collects 
school data (e.g., finances, faculty, curriculum, advertising) and copies of certificates of 



Organization of Postsecondarv Vocational Education 

Montana has five vocational/technical centers enrolling about 
2,700 students. Additional vocational programs are located in three 
community colleges and one four-year institution, which have a combined 
enrollment of approximately 2,300 students. Vocational/technical 
programs in the state cf Montana were reorganized in 1987. On July 1, 
1989, the Board of Regents was given governance over the vocational/ 
technical centers. 

The Professional and Occupational Licensing Bureau of the 
Department of Commerce oversee* 31 proprietary schools but its main 
function is to oversee small businesses. Cosmetology schools are 
overseen by the Cosmetology Board. 

Public Schools 

As a result of the reorganization of vocational/technical 
programs, the state is in a period of transition regarding the 
collection and reporting of data. A transition group is developing 
program evaluation standards—a set of common criteria on which 
programs will be evaluated. Areas such as available resources, faculty 
qualifications, student evaluation, enrollment, completion, and 
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placement are either under r ■> s1 deration or are being actively 
developed. A manual for standardizing data collection and the 
reporting of Information on enrollment, placement, and follow up was 
piloted 1n fall 1989; this manual Is currently under revision. 

Proprietary Schools 

The state collects copies of the certificates of completion from 
proprietary schools but does not count them. No placement or earnings 
Information 1s collected. Schools are required to provide information 
on their financial condition, faculty qualifications, curriculum and 
course outlines, length of courses In relation to cost, and 
advertising. 



103 

110 



Nebraska 



Nebraska presently does not collect any measures of performance from the community 
technical colleges, although recent legislation requires the collection, once every five yearn 
of number of graduates per program and credit hours for each faculty member. This Is test 
information than collected in the past; The state collects completion and placement data 
from proprietary schools, but not rates, and breaks the data down by institution and 
program. Pending legislation and a constitutional amendment could change data collection 
processes for both public and proprietary schools. 



Organization of Postsecondarv Vocational Education 

Nebraska's Postsecondary Education system is comprised of 24 
institutions serving approximately 108,156 students. It is organized 
under several autonomous governing entities: the multi -campus 
University of Nebraska system overseen by its Board of Regents, 7 
public community technical colleges managed in seven administrative 
units, and 15 private four-year colleges and 2 two-year colleges 
governed by the Board of Trustees of Nebraska State Colleges. Six of 
the community technical colleges are administered under locally elected 
School Boards. The seventh is an agriculture technical college under 
the University of Nebraska at Lincoln. All together they serve 
approximately 31,482 vocational students. 

The state's 50 postsecondary proprietary ^schools serving an 
estimated 10,639 students are overseen by the State Board of Education, 
and licensed by the Department of Education's Vocational Education 
Divisiii, 
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Public Schools 

The state does not collect measures of performance from the 
community technical colleges. However, recent legislation requires the 
Coordinating Commission for Higher Education to conduct individual 
program reviews every five years for appropriations purposes. The 
stipulated review criteria limits data collection to two areas: the 
number of graduates per program and the number of credit hours 
generated annually by each faculty member. Collected information 1s 
analyzed, conclusions drawn, and the Commission's program review report 
submitted to the legislature for use in funding decisions. As the 
Commission has no oversight authority over these schools, no site 
visits of any kind are made. Information is submitted on standardized 
forms prepared by the Commission. Any apparent discrepancies in the 
data are corrected through telephone calls. 

A newly passed constitutional amendment, subject to voter approval 

In the fall 1990 general election and effective January 1992, would 

greatly dilute the autonomy of the public governing boards by giving 

the Coordinating Commission for Higher Education greater regulatory 

authority over public postsecondary education. The Commission would 

have authority over the coordination of all public postsecondary 

educational Institutions in three key areas: 

o The development and adoption of a comprehensive statewide 
plan for postsecondary education. 

o Approval of all programs and capital construction projects 
for compliance and consistency with the comprehensive plan. 

o Approval of all budget requests of the local boards and any 
future governing board the legislature may establish for 
oversight of public postsecondary educational institutions. 
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The constitutional amendment also would permit the legislature to 
choose between a single board or an association representing the 
community colleges as the means of coordinating these schools. 

Prior to 1985, the state collected and maintained performance 
outcomes information in the Vocational Education Data System (VEDS). 
Among the outcome information collected were completion and placement 
data, relationship of placement to training, earnings, and gains in 
basic and job skills. A completer was anyone who was either awarded 
the associate degree or certificate, or completed a training program. 
Completion rates were not calculated. PI? anient was defined as any 
completer subsequently employed, regardless '»>e relationship of the job 
to training. However, the state kept separate counts of those placed 
in related and unrelated jobs, as well as those unplaced. Completers 
were surveyed for placement and initial wage levels one year after 
graduation. Through the survey data were collected on completion, 
placement, and earnings, which were categorized by gender, 
racial/ethnic group, and handicap. 

Information on gains in basic or job skills and relationship of 
placement to training also were obtained in the past through employer 
surveys conducted simultaneously with the completer placement surveys. 

The state made annual site visits to participating institutions to 
verify data accuracy and help the schools organize data bases. 

Statewide figures were compiled and disseminated in an annual 
report distributed to the state legislature, the U.S. Department of 
Education, and the general public. 




106 



113 



These efforts may be revived in whole or part with potential 
provisions under the proposed state constitutional amendment. 
According to the Commission's executive director, an institutional 
accountability report incorporating some measures of performance may 
become a reporting requirement. However, this would necessitate the 
development of a statewide accountability program, which potentially 
would take time to finalize. 

Proprietary Schools 

The state's 50 proprietary schools include private vocational/ 
technical institutes. The state collects completion and placement data 
only. This is done annually on standardized forms prepared by the 
State Department of Education's Vocational Education Division. 
Additionally, the schools are encouraged to conduct follow-up surveys 
of completers to gauge relationship of placement to training and 
earnings levels. 

Completion and placement rates are not calculated. The state 
counts the total number who completed each program based upon award of 
diploma or certificate, and breaks down the data by institution and 
program. Placement is strictly defined as employment in the area of 
training. The assessment is done by the schools one year - . ier 
graduation through a state-prepared standardized form, which includes 
total counts of students placed in training related and unrelated jobs. 
Data are collected on initial placement only, but schools are urged to 
voluntarily conduct additional follow-up surveys at three and six 
months intervals after that. Placement data are broken down by 
institution and program. 
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Schools are also encouraged to survey completers for job 
satisfaction, training relatedness, and earnings levels. The 
information is not collected at state level but is reviewed during site 
visits. Success rates In career licensing tests are not collected as 
most participating schools do not have programs requiring career 
licensing as a condition of employment. 

The Office of Proprietary Schools in the Division of Vocational 
Education reported that understaffing precluded annual site visits to 
all participating institutions. However, problem schools are visited 
regularly to review administrative and other records. Data accuracy is 
monitored through interviews with administrative staff, faculty, and 
students during the visits. Noncompliance with information 
requirements is reportedly very low; the only data inaccuracies found 
are attributable to a school's genuine misunderstanding of definition 
of terms under the reporting requirements. 

The data collection efforts are aimed for both consumer rights' 
purposes and management decisions. The state disseminates the 
information 1 a biannual report distributed to the legislature and the 
general public. The report lists all licensed proprietary schools, 
programs offered at each school, plus enrollment, completion and 
placement data. 

A 1978 legislative action made proprietary students eligible for 
state financial aid. However, a 1985 state Supreme Court ruling 
limited state aid to students attending nine-month and longer programs. 

State accreditation is done every three years. The statute 
requires the schools to submit a self -study report and a plan for 
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future growth. The self -study report must include an assessment of 
completion and placement rate, faculty qualifications, and the school's 
financial condition. The Institutional growth plan must Include 
Identified ways to enhance faculty qualifications, Increase student 
enrollment and retention, and Improve facilities and equipment. On 
occasion a school has been closed down for statute violation. 

Pending legislation is expected to generate new policies designed 
to Increase state authority over proprietary schools. Its provisions 
give the State Board of Education the latitude and some funding to 
establish new regulatory standards, such as minimum faculty 
qualifications, policy requirements on tuition refunds and consumer 
rights information for potential students, and specific recruiting 
guidelines to minimize false advertising and other student abuses. The 
LIU was sponsored by the proprietary schools themselves and received 
overwhelming support by the legislature, regulatory agencies, and trade 
and industry associations. 
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Nevada 



Nevada does not collect any measures of performance from Its community colleges, although 
each campus must submit an assessment plan that involves a multiple assessment approach. • 
Annual completion rates are the only measures collected from proprietary schools. 



Organization of Postsecondarv Vocational Education 

Nevada has four community colleges with a total of approximately 
32,500 students. The Postsecondary Education Commission oversees the 
109 proprietary schools in Nevada, 13 of which offer degrees. About 
11,500 students attend the proprietary schools. 

Public Schools 

No measures of performance are collected from the community 
colleges. The Board of Regents has passed a policy that by the spring 
of 1990, each campus must submit an assessment plan that Involves a 
multiple assessment approach. The campuses have been asked to work 
together and utilize a common approach whenever possible. 

Proprietary Schools 

Annual completion rates are the only performance measures 
collected from proprietary schools. The calculation is based on the 
number of students enrolled and the number completing the program. A 
completer is defined as one who completes 75 percent of the school 
program. No distinction is made between full-time and part-time 
students. The information is used in making management decisions. 
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New Hampshire 



Technical colleges in New Hampshire report the number, but not rates, of graduates. 
Placement rates are calculated through student surveys, with most students indicating 
starting salaries as welL three* and five-year follow-up surveys on earnings will be done 
with results reported by program. Hie state does not collect any data of any kind from 
' ;'• proprietary schools. }'. ■ 



Organization of Postsecondarv Vocational Education 

The Department of Postsecondary Technical Education oversees 
technical colleges with programs leading to two-year degrees. The 
seven technical colleges have 2,700 full-time day students and about 
20,000 mostly part-time students in the Division of Community 
Education. The state does not have an oversight board for private 
schools. 

Public Schools 

The technical colleges provide the state with the number of 
students who graduated from a certificate, diploma, or degree program, 
but completion rates are not calculated. Placement rates are 
calculated, and the total number of graduates is compared to the number 
employed, the number not looking for jobs, the number transferred to 
another school, and the number entering the military. Students are 
surveyed just before graduation in June; those without jobs # are 
followed by mail and direct contact until November. The state is 
beginning to consider collecting information on attrition and financial 
aid. 
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Students are asked about starting salaries. Approximately 80 
percent return this Information. For the first time, the state 1s in 
the process of doing a three- and five-year follow up on earnings. 

Schools use state forms to submit the Information, which 1s 
collected once a year. Analyses of placement rates and earnings 
Information are done by program. Data are arranged Into graphs, 
charts, and tables for Individual colleges and systemwide summaries. 

A report 1s prepared at the state level for the Governor, the 
legislature, rnd the school system. The data are used to determine the 
viability of programs and efficiency of operation; they are not used 
for consumer rights purposes. 

Each of the schools compiles a brochure that contains school 
specific Information. The brochures are given to students at admission 
and used for marketing. Copies are provided to libraries and 
businesses and Industries in the state. 

Proprietary Schools 

The state does not collect data of any kind about proprietary 
schools. 
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New Jersey 



Data collection efforts from community colleges in New Jersey are just beginning. 
Graduation rates are collected, but not using a uniform definition. Pre* and posttest results 
on the reading, writing, and mathematics subtests of the New Jersey College Baste Skills 
Placement Tests are reported by institution and program Annually, proprietary schools 
report new enrollecs, dropouts, completers, and placement rates by program area. 



Organization of Postsecondarv Vocational Education 

The 19 community colleges In New Jersey are overseen by the 
Department of Higher Education. The Office of Private Vocational 
Schools in the Division of Vocational Education is responsible for 230 
private vocational, technical, and business schools with nondegree 
programs. Some of the approximately 45,000 students who attend these 
programs earn certificates. Schools of nursing, truck driving, casino 
dealing, real estate, cosmetology, and barbering are overseen by their 
Individual boards or commissions. 

Public Schools 

The state is just starting to collect Information on graduation 
rates from community colleges. At this point the measures are not well 
defined. The state also 1s considering collecting data on retention. 

Every community college student must take the New Jersey College 
Basic Skills Placement Tests 1n reading, writing, and mathematics. 
Both pre- and posttests are given. Analyses are done by college and by 
program. An annual report 1s submitted to the state, and provided to 
the Board of Higher Education, the press, high schools in the state, 
and the general public. Disclosure also 1s made to students and 
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potential students. The data are used 1n making management decisions 
and, to some degree, for consumer rights. 

Proprietary Schools 

Once each year the private schools report new enrollments, 
dropouts, and completers. Both full -time and part-time students are 
included in the count. Placement rates, defined as a percentage of 
completers, are calculated by program and for the school as a whole. 
The information is provided in written format to the state which 
compiles it. into a large report. Analysis is done by school and by 
program. The reporting is done about four to six months after the data 
are collected. The data are used for some management decisions and to 
keep the public informed. No regular monitoring of compliance or 
accuracy is conducted. However, any information used in an advertising 
brochure must be verified. 
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New Mexico 



In New Mexico* pweecondary vocatkmal/teclmkal institutes report information on 
completion and placement rates, relationship of placement to training, and earnings 
information. Much of this information ii collected through required annual follow-up 
surveys of completeri/letven and tlwir emptoyere. Placement data are summarised by trade, 

program that mandates collection of performance measures. The state does not collect any 
performance measures from proprietary schools, although enrollment and completion totals, 
by program, are included on annual license application forms. 

s 

Organization of Posts^ondarv Vocational Education 

The New Mexico Commission for Higher Education is the coordinating 
board for the state's public and private postsecondary and higher 
education Institutions. Thus, the Commission oversees 112 proprietary 
schools and 29 public and nonpublic postsecondary and higher education 
institutions. Of the latter, 23 are public colleges with 6 four-year 
colleges (one of which offers two-year certificate programs) and 17 
two-year institutes that offer a wide range of academic transfer, 
vocational -technical education, and community service programs. The 
other six are nonpublic colleges: 4 four-year colleges and 2 two-year, 
Native American institutions. All together the institutions serve 
approximately 70,000 students. Among these, 22,000 attend the two-year 
colleges with roughly 13,000 enrolled in the vocational/technical 
programs . 

The state's 112 proprietary schools are licensed by the Commission 
on Higher Education and governed under the 1978 licensure legislation. 
The Commission has limited oversight responsibility for proprietary 
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institutions. Data on the number of students attending those schools 
is not available. 

Public Schools 

The Vocational -Technical and Adult Education Division of the State 
Department of Education collects measures of performance from the 17 
postsecondary vocational/technical institutes, including completion and 
placement rates, relationship of placement to training, and earnings 
Information. The Division obtains a hard copy of school -provided data 
for the various analyses. The completer is either the recipient of a 
certificate, diploma, the associate or full degree, or, for the 
vocational/technical students, completion of a whole sequence of 
courses for a two-year program. A standard definition of a completer 
currently is being developed. The completion rate is computed for 
full-time graduates only. 

The State Department of Education has defined placement as the 
full-time or part-time completers employed in their area of training. 
To determine placement rates, relationship of placement to training, 
and earning levels, the schools are required to conduct annual follow- 
up surveys of completers/leavers of vocational -technical programs and 
their employers. Information collected by survey questionnaires is 
organized by program under ten categories: those who sought 
employment, found employment, were unemployed, were unemployed seeking 
work, obtained employment, were continuing education, were not In the 
labor force, had unknown status, average hourly wage earned, ai.d 
av age employer rating. The data are then aggregated under three 
categories: those available for employment, those currently employed, 
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and the number unable to find work. The placement rate is computed as 
the percentage ratio of the total number employed to the total 
available for employment for a given academic year. Placement 
Information Is summarized by trade, gender, racial, and ethnic groups. 

The annual employer satisfaction survey helps to gauge gains In 
Job skills and the relationship of placement to training by asking the 
employer to rate the completer's job performance on a 5-point scale. 
The employer responds to the question as to whether employee job 
performance at entry level is commensurate with two-year training. The 
average employer rating 1s computed for full-time employees only. 
Gains in basic skills are not collected at state level, although the 
adult basic examination (ABE) is routinely administered at entry for 
adult programs. Earnings information is collected on entry salary 
levels and computed for average hourly wage and average annual 
salaries, for full-time employees only. 

Site visits are conducted annually to assure compliance with 
education standards, for instance, review of recruitment process, 
student services, and others. Site visit reports are supplied to 
participating institutions and remediation of deficiencies enforced. 
The Commission on Higher Education uses independent auditors to verify 
funding -related information for its report to the legislature. 

The state produces an end-of-the-year report from hard copy of 
school -provided Information, used only for the Division's management 
decisions. The analyses and evaluative interpretation of the school 
conducted follow-up studies 1s compiled into an annual vocational/ 
technical education follow-up report available to the public upon 
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request. This report does not include student loan default rates as a 
measure of performance, and neither is that information a reporting 
requirement of the schools. However, the state follow-up report does 
contain information on the type of financial aid students received, for 
Instance, basic grant, loans, college work study, social security, and 
others. 

Recent legislation promises to expand and enhance the quality of 
education at two-year institutes as part of a statewide overhaul plan 
to improve the participation rate in higher education. The legislation 
mandates the Commission for Higher Education to develop a statewide 
five-year planning process, including institutional planning guidel ines 
under which aforementioned measures of performance would be collected 
and projections for improvement and growth made on five-year basis. 
The Commission will assist each institution to develop its own five- 
year plan to assure consistency with mandated statewide guidelines, and 
suggest annual updates accordingly. 

Within the statewide planning process the legislation also 
establishes the institutional accountability program under which 
measures of performance will be collected. These will include 
completion and placement rates, relationship of placement to training, 
and gains in basic or job skills. The Commission will develop a 
statewide criteria for assessing educational outcomes and will 
determine specific assessment indicators. The state will assist each 
school to develop its own assessment procedure that suits its 
informational and program needs, for instance, retention studies, 
research activities such as follow-up studies of graduates, and 
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assessing successful students in either continuing education or 
employment. The institutional accountability report will be submitted 
in hard copy as part of the five-year plan. 

Follow-up studies of graduates are still to be used to generate 
placement information. The data will be summarized under five 
completer categories: type of employment, military service, continuing 
education and location, unemployed, and status unknown. An added 
dimension of the follow-up study will be the assessment of flight from 
state over time by discipline or trade and suggestions for plans to 
reverse the trend. 

The statewide plan requires site visits during which institutional 
plans will be reviewed. 

The data collected will be used in funding decisions, initiating 

new programs, etc., as well as for consumer rights purposes. The 

information will be published in the following Commission reports. 

o The institutional accountability reports will be compiled 
<nto the Annual Report of the Sta tewide Assessment Results. 
In addition, legislation requires the institutions to publish 
in a community newspaper their summaries of achievements and 
expenditures. 

o The Statewide Plan for Higher Education will be made 
available for review by the legislature, the Governor, other 
institutions, and the general public. 

o Institutional plans will be made available for review by 
other Institutions updating or Implementing their own plans 
for new program development and resource allocation. 

These legislative initiatives followed findings of a mandated 

Investigation of the Commission on Higher Education that revealed on a 

macro level that New Mexico's participation in higher education was 

lower than national averages. 
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Proprietary Schools 

No measures of performance are collected at the state level for 
proprietary schools as these data are not required under the licensure 
law. However, the schools annually submit enrollment and completion 
totals, as well as the new courses offered, as part of the license 
application. These are submitted on standard forms prepared by the 
Commission. Completion data are collected by Institution and program. 
A completer is regarded as anyone awarded a certificate of completion. 

Site visits are limited to problem schools due to staff shortage. 
The state generally relies on the integrity of school official for data 
accuracy. 

The data collected are used to compile the annual state directory 
of proprietary schools. It lists the number of proprietary schools 
statewide and their business status (licensed, unlicensed, exempt, 
accredited) and programs offered. The directory is widely disseminated 
to the public. Investigative reports on problems schools are available 
to the public on request. 

State efforts to oversee proprietary schools have been very 
frustrated due to the limitations of the 1978 statute. For instance, 
the New York Technical Institute recently suffered financial 
difficulties and closed down Its site in Santa Fe. New Mexico students 
suffered tuition losses, which include payable Pell Grants and other 
student loans. This and other well-publ icized experiences have 
prompted the Commission on Higher Education to sponsor legislation 
seeking greater state regulatory authority over proprietary schools. 
An advisory council is revising the proposed legislation for 
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effectiveness. Meanwhile, the Commission 1s developing a performance 
outcome reporting system for Implementation under the proposed 
legislation. 
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New York 



In New York, IPEDS was used as a guide tot dMcoUection from public schools. 
Enrollments aiid'ftumi^ annually by sex and race£thnt<% Also 

reported are sclent loan detoult rat^^^^i^, tnd school facilities. There are mre 

the state collects information m .itii IlWttftejt of completers, placement rates, and whether 
placement is related to training. : W proprietary schools are required to publish their 
enrollment, completion, and placement retain their catalogs. 



Organization of Postsecondarv Vocatio nal Education 

The Bureau of Academic Programs Review oversees all colleges that 
award degrees, including associate degrees. The 124 schools consist of 
30 public community colleges, 13 other public colleges, 26 proprietary 
degree granting schools, 22 two-year independent nonprofit schools, and 
33 other independent nonprofit colleges. Together they enroll about 
260,000 students. 

The Division of Proprietary Schools oversees all postsecondary, 
nondegree-granting institutions, including licensed technical and trade 
schools. About 360 schools enrolling approximately 150,000 students 
are in this category. 

Public Schools 

The Integrated Postsecondary Education Data System (IPEDS) was 
used as a guideline for the data collected by the state. Col luges and 
universities must provide degree-credit enrollment for full-time and 
part-time students and categorize these students by gender and 
racial/ethnic status. Schools also must provide noncredit enrollment. 
For students in occupational ly specific programs, enrollments must be 
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shown by racial/ethnic category and sex of student; IPEDS codes are 
used to Identify programs. The state also collects the number of 
degrees and certificates conferred by each school. Graduation rate 
data for full-time, first-time freshmen and full-time undergraduate 
transfer students by racial/ethnic category are collected by the state. 
The data measure retention patterns of students and indicate the amount 
of time needed to complete undergraduate degree programs, including 
associate degrees. The state does not collect placement rates, 
earnings information, or basic skills data. 

The state collects enrollment and completion data annually. Every 
two years all schools must complete an extensive questionnaire on 
topics ranging from recruitment of women and campus climate to finances 
and facilities. The data are entered into a computer at the Department 
of Education. Enrollment and number of degrees awarded are included in 
a number of reports prepared by the State Education Department. This 
information is used by both the secondary and postsecondary schools. 
Members of the public can obtain information by calling the Department 
directly. 

The state makes site visits to the schools for auditing and 
accreditation purposes. The state comptroller audits the 
administration of the state and federal aid programs. As a part of the 
accreditation process, schools must submit a self-study report. If the 
schools conduct any surveys of graduates or employers, they are to be 
included in the report on the self -study. Visiting teams verify the 
information in the report. If a program does not meet accreditation 
standards, the Bureau of Academic Programs Review has the authority to 
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terminate 1t. Other penalties that have been used are banning new 
admissions to a program and completely shutting down a school. 

The Bureau requires schools to provide Information on student loan 
default rates. Faculty quality and library and laboratory facilities 
also are monitored by the state. 

Proprietary School s 

The state collects the number of students who were awarded a 
certificate or diploma following program completion; this number is 
subdivided by gender and full-t**e and part-time students. Placement 
rates are collected for completers. Categories of placement are 
related placement, slightly related placement, unrelated placement, 
military employment, seeking employment, pursuing further education, 
other/unavailable for employment, and status unknown. 

The data are aggregated and standardized by the Division and made 
available to the reporting schools. The schools are required to 
publish enrollment and completion and placement rates in their 
catalogues. The accuracy rate for enrollment and completion 
information was said to be high, but the accuracy rate of placement 
rates is low. The state conducts random spot checks on schools 
suspected of or known to report false placement data. Schools found 
reporting false information receive heavy penalties, such as closing 
down the programs for which false data were given. 

The information is used by the Governor and the legislature, by 
the post secondary schools for policy making, and by secondary schools 
for student counseling. 
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North Carolina 



The only data reported by community colleges in North Carolina is number of completers, 
reported by program. New adoption of "27 Measures of Effectiveness* will require school- 
wide reporting of additional data, including number of students retained from previous 
quarters and employer satisfaction with graduates/ Proprietary schools presently report only 
enrollment numbers at the beginning of each year. 



Organization of Postsecondarv Vocational Education 

The North Carolina Community College System consists of 58 
community colleges, which enroll about 218,000 students in their 
curriculum programs and 272,000 students in their occupational 
extension programs. Curriculum programs are technical or college 
transfer programs leading to a degree, certificate, or diploma. 
Occupational extension programs do not lead to a credential; some of 
these programs may be designed for a particular employer. Private 
schools involving licensed occupations, such as cosmetology, barbering, 
and truck driving, are administered by the individual licensing boards. 
The Board of Community Colleges approves the remaining 42 proprietary 
schools, which enroll approximately 6,000 students. 

The Education Assistance Authority provides financing for 
students, generally in the form of loans. It has no direct involvement 
in licensure, accreditation, or the measurement of performance. 

According to the Governor's Education Advisor, the Southern 
Association of Colleges and Schools (SACS) has influenced the types of 
measures collected by the state. SACS, an accrediting body for both 
public and nonpublic schools, has changed its accreditation 
requirements from input to outcome measures. 
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Public Schools 

The community colleges collect Information on the number of 
completers. A completer is anyone who receives a degree, certificate, 
or diploma. The state is discussing measuring progress of students 
toward completion and the retention of students. The state does not 
collect placement rates or earnings information. 

Schools provide the data on computer tapes on a quarterly basis. 
Analysis is done by school and by program. 

The data are provided to the Governor's education advisor, the 
legislature, and the schools themselves. The data are used at the 
school level for planning purposes and at the state level for budget 
allocations. The community colleges are 85 percent subsidized by the 
state. 

In July 1989, the State Board of Community Colleges adopted "27 
Measures of Effectiveness," which will be used to measure the community 
college system as a whole rather than individual schools. In selecting 
the measures, an emphasis was placed on ones that could be reported in 
1990 with data that already were being collected. However, some of the 
measures will require additional data collection; these Include the 
number of students retained from previous quarters and employer 
satisfaction with graduates. 

Proprietary Schools 

The proprietary schools are required only to provide enrollment 
numbers at the beginning of the year. A legislative research committee 
is studying proprietary schools to determine if laws need to be 
strengthened. 
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North Dakota 



In North Dakota, the state collects enrollment, completion, and placement data from public 
vocational education programs. They also report on curricula and program*. Proprietary 
schools report number of completers and school characteristics in their annual license 
renewal applications and are requested, hut not required, to submit placement and earnings 

Organization of Postsecondarv Vocational Education 

The Department of Vocational Education oversees vocational 
programs in 3 community colleges, 6 four-year colleges, and 2 two-year 
institutes. The Department also is responsible for licensure of 11 
proprietary schools but has no authority over cosmetology schools. 

The North Dakota legislature meets biannual iy; it will be back in 
session in 1991. The Interim Legislative Committee on Education did 
not receive any request for legislation addressing the performance 
outcomes issue. 

Public Schools 

The Department of Vocational Education collects enrollment and 
completion data from vocational education programs. A completer ;s 
defined as one who completes either a planned sequence of courses or 
all the competencies for the program. Data are collected by race and 
sex. A follow-up survey is required six months after program 
completion. Placement information is divided into six categories: 
employed in a field related to training, emplt ;ed in a field unrelated 
to training, employed by the military, continuing education, 
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unemployed, and status unknown. Earnings Information was collected in 
the past but 1s not collected currently. 

Each year the Department of Vocational Education sends schools a 
computer print-out listing all programs. The schools enter information 
on enrollment, completion, and placement on the printouts and return 
them to the Department. A report on enrollments, completers, number of 
schools, programs, and curricula is submitted to the State Board of 
Higher Education. The information primarily is used by the 
postsecondary schools themselves to review enrollment changes over 
time. The data are not normally disseminated to students or the 
general public but are available on request. 

The Higher Education Department collects enrollment and completion 
information from the community colleges and maintains the data in a 
central mainframe computer system. Completers are those who have 
received an associate degree, diploma, or certificate. Completion data 
are reported annually and enrollment data are reported each term; this 
is considered primarily a procedural exercise. A booklet with programs 
offered in the community colleges and a list of graduates in each field 
of study is compiled by the Department and made available to the 
general public. The Higher Education department does not collect 
placement or earnings Information. 

Proprietary Schools 

In the annual license renewal requests, proprietary schools must 
include the number of completers. Placement and earnings information 
is requested but not required. To be approved as a business, the 
school must submit a license application containing the source of 
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financial backing, faculty qualifications, a 11st of agents, and 
bonding procedures. Information from the applications 1s summarized 1n 
a report to the State Higher Education Board. 

Each school receives an annual site visit by the Department of 
Vocational Education. Activities during the visit Include the 
verification of reported data, checking compliance with legislation, 
and follow up on student complaints. A financial review of schools 1s 
done annually. If a school does net meet approval requirements, it 1s 
closed down. 
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Ohio 



Ohio collects information on enrollment and completion, placement rates, and initial wage 
rates torn public jxwtsecondary schools. Individual instructors arc responsible tor collecting 
the annual Mow-up information on completers/leavers, including current educational and 
employment status, current safety, and relationship of employment to training. New 
requires proprietary schools to report completion and placement rates. 




Organization of Postsecondarv Vocational Education 

State responsibility for postsecondary vocational/technical 
training in Ohio is spread across several government agencies. Contact 
was made with the State Board for Proprietary School Education and The 
Division of Vocational and Career Education in the Department of 
Education, which is responsible for public programs for part-time and 
full-time adults and Perkins Act funded apprenticeship training. In 
spite of multiple attempts to contact the Board of Regents, information 
for full-time for-credit programs was not able to be obtained. 

There is s fairly large vocational/technical education sector in 
Ohio. Some 209,000 adults are enrolled in supplemental programs, 
70,000 adults are enrolled in full-time programs for displaced workers 
and those making mid-career changes, and over 6,000 are enrolled in 
Perkins Act funded apprenticeship programs. There are also 365 
proprietary and 67 private non/profit postsecondary vocational/ 
technical schools in the state. 

Ohio collects information on enrollment and completion, placement 
rates and initial wage rates. Recent legislation extended information 
requirements to the proprietary schools. The responsibility for 
collecting the follow-up information on completers is given to 
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Individual instructors. Their Information is assembled by the schools 
but is reported to the state by instructor. 

A special longitudinal study of secondary level vocational/ 
technical programs is being conducted by the Department of Economic j at 
Ohio University. The study design calls for the use of IRS data 
matched with the social security numbers of those enrolling in 
vocational courses. 

Public Schools 

The major thrust of state information collection is an annual 
follow-up survey of program completers and leavers. Those ending 
participation in a program during a July through June annual cycle are 
contacted the following February or March. The follow-up survey 
requests information on current educational and employment status, 
current salary, and whether employment is related to the training 
received. 

The follow-up survey is administered by the instructors using a 
standard state-supplied form. Completers of full-time programs as of 
June are surveyed in February or March of the following year. 
Information is aggregated for each instructor and each program, and 
reported to the State Division of Vocational Education. 

The placement results from the follow-up survey are published in 
a booklet that is sent to all school districts, the State Board of 
Education, and, on some occasions, to all members of the General 
Assembly. The information also is provided to the schools and 
instructors. The schools may decide to use the information for local 
marketing efforts. Information on wages is not published. The major 
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analysis of the data conducted by the state 1s a comparison of the data 
from these programs- with general employment data. 

Compliance with the Information requirements has been very good, 
as participation 1s required for continued funding. An on-site review 
of the data and the procedures used to collect it is conducted every 
five years. During the review, a sample of individual data collection 
forms is examined, and some graduates are contacted. 

One use of the information by the programs is in demonstrating 
that they are marketable. Schools must show that each program is able 
to obtain training-related placements for its graduates, or the state 
may require that the program be discontinued. The state provides the 
individual school affected with the names of programs and instructors 
that do not achieve sufficiently high achievement rates. 

Proprietary Schools 

New state legislation has revised the requirements for measuring 
the performance of proprietary schools; completion and placement rates 
will be among the measures obtained. Rules for collecting the data 
will be ready in March 1990. Items contained in the state's default 
management plan are likely to be the core of this effort, as the 
schools will be required to collect this information anyway. 

An annual report on proprietary schools has been sent to the 
Governor, the legislature, and libraries and other state agencies; 
copies are available for the individual schools on request. The report 
has been based on a survey that collected information on the number 
enrolled, the number of withdrawals, the number of completions, and the 
number of placements. Individual schools were free to develop their 
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own definitions of these terms. The survey has not taken place 1n two 
years. 
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Okl ahoma 



For recipients of associate degrees, community colleges in Oklahoma submit data on 
semester-based enrollment and completion data for at the program level, which allows for 
calculation of retention-dropout rates, freshman progress analysis, etc Negotiations are 
under way to access unemployment insu*nce wage data and employer files, 
Vocational/technic 

on completion, placement, relationship of placement to training, and gains in basic and 
occupational skills. Completion and placement data are broken down by gender, race, ; 
handicapping conditions, and other minority groupings. The only performance outcome 
data collected from proprietary schools are enrollment and completion data as part of 
licensure requirements. V. ... ■), \ / 



Organization of Postsecondarv Vocational Education 

The Oklahoma Regents for Higher Education oversees the state's 
degree granting postsecondary institutions. The agency defines the 
general term for all standardized core curricula, accredits the 
associate degree granting public and proprietary schools, and also 
serves as the state's official student loan guarantee agency. The 
Regents also holds limited oversight responsibility for the four 
postsecondary education Centers. 

The state has 242 postsecondary and higher education institutions 
serving an estimated 191,000 students. The public sector is comprised 
of 77 institutions that include 12 four-year colleges and universities 
offering associate degrees, 13 community colleges, 4 Centers, and 48 
Vocational/Technical Area Schools. The area vocational/technical 
schools are overseen by the Department of Vocational and Technical 
Education. They serve an estimated 20,000 daytime, full-time students 
plus thousands of others who take only brief series of courses to 
upgrade their skills on part time basis. Adult programs also are 
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offered, hence articulation at the Area Centers 1s at both secondary 
and postsecondary levels. 

The nonpublic Institutions Include 150 proprietary schools and 15 
private Independent colleges. The Oklahoma Private School Board 1s the 
licensing agency for the state's 150 proprietary schools. Among them, 
30 are based out of state; 25 are nationally and regionally accredited. 
Accreditation makes the students eligible for state financial aid 
programs. The entire proprietary sector serves an estimated 21,000 
students. 

Public Schools 

Community Colleges . The Regents of Higher Education 
systematically collect few measures of performance from the 13 
community colleges. Semester-based enrollment and completion 
information to conduct a variety of Institutional assessment analyses 
such as retention-dropout rates, freshman progress analysis, fiscal 
studies, and others for a variety of audiences. Data for retention- 
dropout analysis are collected two months after the semester ends and 
the freshman data two months after students complete their first year. 
Graduation data are submitted immediately after program completion. 

Completers are defined as the recipients of associate degrees. 
The assessment Information 1s submitted in computer tapes and 
maintained in a centralized data base at the Regents office. This 
provides the capability to collect additional measures of performance 
and conduct further Institutional and student analyses as needed. 
Negotiations are under way for the Regents to be authorized access to 
the state wage data and employer files for use in student analyses. 
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Placement rates are collected at the program level. Gains 1n 
basic skills are assessed through entry and exit testing using a 
standardized test such as the ACT scores at admission and program- 
specific exams at completion. At entry students under 21 years of age 
must have acceptable scores in standardized tests prior to admission, 
but adults can be exempted from this. 

Institution-specific retention/dropout study reports are 
distributed to respective participating schools. Studies of the 
progress of freshmen are sent to high schools as indicators of how 
their graduates are doing. These reports also are made available to 
the public on request. The assessment information also is used to 
compile a variety of specialized reports used for agency's internal 
management decisions. 

The Regents centralized data base preempts the immediate need for 
site-visit verification. The current compliance and accuracy 
monitoring procedure amounts to each institution's review and 
correction of its own draft analytical profile as prepared by the 
Regents for Higher Education each semester. However, a policy 
initiative on retention rate studies is expected to require team site 
visits to verify schools compliance with minimum grade point average 
requirements and other data. 

Oklahoma is reportedly experiencing an increase in community 
college enrollment prompted by the increases in university tuition and 
admission standards. 

Vocational /Technical Schools. Performance outcomes, annually 
collected by the area vocational/technical schools only on the full- 
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time students, Include data on completion, placement, relationship of 
placement to training, and gains 1n basic and occupational skills. 
Each area voc/tech school defines Its completer, generally someone who 
has completed an occupational program of study. Placement data are 
generated through follow-up surveys of completers conducted by the 
instructors nine months after the school year ends. Completers are 
grouped in terms of related or unrelated placement, military, 
continuing education, and other categories. Completion and placement 
information 1s broken down by gender, race, handicapped students, and 
other minority groups. Occupational competency tests, administered on 
optional basis using assessment instruments developed by the state, are 
used to assess student gains in basic and job skills. Test results 
also are used as ndicators of program quality. 

Performance outcome data are collected in hard copy and maintained 
in a central electronic data base. The effort 1s aimed at both 
management decisions and consumer rights purposes. Reports compiled 
from the school -provided information are distributed to the Governor 
and the legislature for appropriations legislation, other state 
government agencies for policy making, as well as to participating 
institutions and the general public. The State Department of Commerce 
and the Office of Civil Rights are among common users of the placement 
data. The state reportedly has no proposed or pending legislation 
dealing with postsecondary vocational programs. 

Site visits are conducted for product and financial audits. 
Program review is done by occupational supervisors using program 
specific evaluation instruments. Schools reportedly have good working 
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relations with the review supervisors. Accreditation reviews are 
conducted every five years by pertinent exter 1 teams. Administrative 
and financial audits are done annually. Verification of school- 
provided data is not done on site visits. 

Proprietary Schools 

Several years ago the Private Schools Association, using its own 
study of economic impact of proprietary schools, successfuMy lobbied 
the legislature that the private schools indus^y was a major economic 
force in the state and deserved an education board of its own. The 
effort resulted in the establishment of the Oklahoma Private School 
Board preempting the oversight role then played by the State Department 
of Education. It also marked the end of state level collection of 
performance outcome information such as enrollment, completion, and 
placement data. 

Currently, enrollment and completion data are collected only as 
part of the licensure requirements. The completer is anyone who has 
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satisfactorily completed all phases of a program. The information is 
collected annually on course-by-course basis. 

The Board also conducts annual site visits during which 
administrative and financial records are reviewed, and facilities and 
equipment inspected. In addition, students are interviewed for their 
satisfaction with programs offered, quality of instruction, school 
recruitment methods M who recruited them, as well as for other 
concerns. Compliance with information requirements is reportedly high 
and relationship with participating institutions positive. Each school 
receives an evaluation report following the site visit. School 
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deficiencies deemed unharmful to student learning can be remedied over 
a 30-day period. Fraudul .it practices can lead to school loss of 
eligibility for student loan programs. School closure and license 
revocation Involve a lengthy litigation process that tends to be tipped 
1n favor of institutions. 

Collected information is published in the annual directory of 
proprietary schools. The publication also lists a broad outline of 
courses offered. 

Presently, there 1s no proposed or pending legislation addressing 
the performance outcome issue with respect to proprietary schools. 
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Community colleges In Oregon report only 1PED$ data to the state. A system to track 
completers through the unempto)inent insurance wage (lata system is under development 
Apart from the norm in other states, proprietary schools are required to provide 
considerably more information to the state than are public schools. Recent legislation 
requires them annually to submit a substantial amount of information, mostly with license 
renewal applications. Included in this required information is enrollments, completions, and 
placements related to training, each by program area. School! are placed on probation if 
their completion and placement rates fell below 7$ percent of the state average. Additional 
required information includes number of students passing licensing exams, school 
characteristics (e.g., faculty, facilities, finances), and copiesof advertising. The state uses 'at- 
risk* indicators to review schools. 



Organization of Postsecondarv Vocational Education 

The 16 community colleges in Oregon enroll approximately 280,000 
full- and part-time students. The state oversees 240 proprietary 
schools that are attended by about 24,000 students. 

Public Schools 

Only IPEDS data are collected from the community colleges by the 
Education Department. Community college students are given a basic 
skills test at the time of enrollment. The Instrument, Basis, was 
developed by the state. It 1s used as a diagnostic and mastery test. 

The Employment Division 1s in the process of developing a follow- 
up system that will track program completers from community colleges 
through the state wage data system. The colleges are providing social 
security numbers of graduates, program of training, CIP code, year 
completed, and demographic data to employment services. The first 
report was scheduled to be released In October 1990. 
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Proprietary Schools 

In 1989, new legislation was passed that requires proprietary 
schools to submit a substantial amount of data to the State Department 
of Education. Much of the Information must be provided with the 
license renewal applications. These are submitted annually based on 
the anniversary date of the each school's license, so they arrive at 
the state office on a staggered basis. At the current time, reporting 
is done in a written format but computerized data reporting 1s expected 
within the year. 

Enrollment and completion information by program is part of the 
data that the proprietary schools must provide. A completer is someone 
who graduated from a program or gained a license or certificate. Both 
full- and part-time students are included, but only a few students 
attend part time. 

Schools also must provide the number of graduates by program who 
have been placed in occupations related to their training. Schools 
provide the name of individual students and their employers to the 
state; the state checks a random sample of these data for accuracy of 
reporting. 

Each year the state will compare the completion and placement 
rates for each proprietary school and the arithmetic average for 
completion and placement rates for all schools in th« state offer ing* 
similar programs within the occupational area. If a school fails to 
maintain completion and placement rates of at least 75 percent of the 
arithmetic average, it will be put on probation for up to one year. If 
the school fails to maintain completion and placement rates of at least 
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75 percent of the average for two consecutive years, the state may 
revoke the school's license. 

Along with license applications, schools must provide the number 
of students passing the state licensing exam, if applicable. The state 
requires information on faculty qualifications, facilities and 
equipment, and school financial condition to be included. Schools must 
submit copies of their advertising. Records must be available to 
document any advertising claims about salary and placement. Schools 
have been required to retract advertisements that touted salary levels 
higher than a recent graduate could expect to receive. 

The state prepares a statistical report with analyses. The 
proprietary schools are informed that the report exists and may receive 
a copy on request. General information files are open to the general 
public, including reporters who make frequent visits. Since 
proprietary schools are private enterprises, some information is kept 
confidential. Information is provided to the State Board of Public 
Education, the executive branch, and the legislature. The state also 
responds to telephone calls from parents and prospective students. 
Individual schools must give all prospective students the following 
Information prior to signing an enrollment agreement: 1) total number 
of students currently enrolled in the program in which the individual 
is enrolling; 2) number of students who enrolled in the program during 
the last reporting period; 3) number of students who left the program 
without completing during the last reporting period; 4) number of 
students who graduated from the program during the last reporting 
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period; and 5) number of those graduates placed or working 1n full- 
time, directly-related occupations. 

These efforts are done primarily for consumer rights purposes but 
some managements decisions are made as a result of them. In addition 
to the data collection, the state makes site visits to investigate 
complaints or problems that have arisen. Schools are inspected on an 
annual basis. 

The state uses a series of "at-risk" 1 indicators that suggest if a 
school 1s in trouble. These indicators include numerous complaints 
filed with the state, high student loan default rate, high staff 
turnover, significant drop in enrollment, low placement rates, and 
possible financial instability or a poor financial report. When some 
of these factors begin to occur in a school, school officials must meet 
with the state in an at-risk interview. If the school cannot rectify 
the problem, the state begins an orderly process for closing the school 
and transferring the students to another program or provides assistance 
in selling the school. 



143 

150 



Pennsylvania 



Pennsylvania ooUects information firom public schools on number of completers (but not 
are completed ty school and program. No data are collected from proprietary schools. 



Organization of Postsecondarv Vocational Education 

The Bureau of Vocational and Adult Education in the Pennsylvania 
Department of Education oversees the community colleges and branch 
campuses of the state university, the vocational/technical schools, and 
proprietary schools. Altogether, 110 of these schools offer vocational 
programs, 85 of which are eligible for Perkins funding. In 1986-87, 
the vocational education and postsecondary programs served about 88,000 
students. 

Public Schools 

Information is not collected on how many students start programs, 
so completion rates cannot be calculated. The state does obtain 
information on the number of completers—those who graduate with a 
terminal degree from an approved program. A positive termination 
placement rate is calculated by comparing the number of completers to 
the number of students employed in related occupations plus those 
continuing their education plus 'hose in the military. 

Analyses are conducted by school and by program. The state does 
not prepare a report on this information. Rather, data summaries are 
made available to a committee that works with the approval of programs 
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and makes funding decisions. The data summaries are available about 
eight months after the data ar° collected. 

Currently, data collection 1s primarily a procedural exercise. In 
the future, the state will be able to defund programs once standards 
have been established. At present there are no standards. 

Before the postsecondary data collection system was Implemented, 
an Individual student follow up was done. This was dropped a few years 
ago. The state has been discussing reimplementing student follow up. 

Proprietary Schools 

No data are collected from the proprietary schools. 
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Rhode Island 



In Rhode Island, community colleges report number of graduates by six placement 
categories. For training-related placements, initial wage rates, employer, and position title 
also are reported by program through annual student mail surveys. For proprietary schools, 
number of graduates (but not graduation rate) are reported as well as placement rates if 
: ' used in advertising. 



Organization of Postsecondarv Vocational Education 

The Rhode Island Office of Higher Education is responsible for one 
community college, which has an enrollment of about 15,000 students in 
credit courses. The Office also oversees 24 proprietary schools with 
a total of about 6,000 students. 

Public Schools 

The community college provides the state with information on the 
number of graduates by program. A graduate is an individual who has 
received a degree or certificate. The number of graduates in each of 
the following placement categories is also submitted: 1) employed full 
time; 2) employed full time and in an educational program part time; 3) 
in an educrtional program full time and employed part time; 4) in an 
educational program full time; 5) employed part time: 6) unavailable 
for employment; and 7) unemployed and searching. For graduates who are 
working in their field of study, initial wage rates, the name of the 
employer, and position title also are provided. The individual 
graduates determine if their jobs are related to their training. The 
information is collected through an annual mail survey of all 
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graduates. A follow-up telephone call is made to graduates who do not 
respond through the mail. 

The information is assembled into a report that has one page for 
each program. The report is disseminated to high schools, employers, 
and state agencies. The report is used to inform students and for 
public relations. 

Information on faculty qualifications is collected at the time of 
hiring and promotion only. 

Proprietary Schools 

Rhode Island collects information on the number of graduates of 
proprietary schools but not how many students started; therefore, 
completion rates cannot be calculated. Placement rates are not 
routinely collected by the state. If a school wants to put placement 
rate information in an advertisement, the school must submit student 
names and telephone numbers to the state. A random sample of the 
students is telephoned by the state to verify placement. No other 
measures of performance are collected at the state level. 
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South Carolina 



Institutional^ 

in for wMeirtion by pubUo postsecondaty schools fti South Carolina. Performance 
components include licensing and certification results, program changes as a result of 
eweroai changes, ^ of entering 

students in meeting ^is^kms prere^ 
schools areenioiuwnt numfcr^ "-'v 



Organization of Postsecondarv Vocational Education 

In South Carolina, the Commission for Higher Education coordinates 
5 two-year colleges, which are governed by the State Board, and 16 
technical colleges, which are governed by the State Board for Technical 
and Comprehensive Education. In FY 1989, the technical colleges 
enrolled about 32,000 FTE students, which represented about 54,000 
individual students. The Commission also licenses the 12 proprietary 
schools offering associate degrees. The 155 proprietary schools with 
programs leading toward a certificate or diploma are licensed by the 
Office of Adult Education in the Department of Education; they enroll 
about 29,000 students. The 53 vocational/ technical schools are under 
the State Board of Education, which does not treat secondary and 
postsecondary students as separate segments. About one- third of the 
107,000 vocational/technical students were estimated as being at the 
pos' secondary level. 

Public Schools 

In 1988, the South Carolina legislature mandated outcomes 
assessments in 18 areas for all two- and four-year postsecondary 
schools. Assessment in these areas, called institutional effectiveness 
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components, is to be phased in over four years; not all areas will be 
assessed annually. The state does not endorse any specific 
methodological strategy for conducting the assessments. The components 
include performance of professional program graduates on licensing and 
certification exams, reports of program changes that have occurred as 
a result of external program evaluations, alumni follow-up studies, 
analysis of undergraduate retention and attrition, assessment of 
administrative and financial processes and performance, and assessment 
of facilities. 

The first report on institutional effectiveness was published in 
January 1990 and was based on data from the 1988-89 academic year. The 
report focused on three components --performance of professional program 
graduates on licensing and certification examinations, reports of 
program changes occurring as a result of external program evaluation, 
and success of entering students in meeting college or university 
admissions prerequisites. The evaluation for each institution was 
reported separately with each school examining its results in relation 
to its own goals. Comparisons across schools are discouraged by the 
state. 

The report on institutional effectiveness is disseminated to all 
institutions, members of the legislature with educational Interests, 
the Governor's office, and members of the general public expressing a 
interest. The thrust of this assessment effort is program improvement. 

The State Board for Technical and Comprehensive Education collects 
data on u.e number of students who received a degree, diploma or 
tificate. Full-time students and part-time students are counted 

149 




separately. The State Board does not collect earnings information or 
basic skills gains. 

The technical colleges currently submit their data on diskette but 
the state is in the process of putting all schools on-line. Site 
visits and telephone calls are made to confirm the data. The board 
uses the data to prepare an annual report that is submitted to the 
Commission on Higher Education only. The information is used for 
program improvements and enhancements; it has resulted in both the 
revision and discontinuation of some courses. 

The State Board of Education does not separate data on secondary 
and postsecondary students attending vocational/technical programs. 
The number of students completing a course sequence 1s collected. 
Vocational/technical programs are required to achieve a placement rate 
of at least 50 percent averaged over a three-year period. Placement 
rate is based on the percentage of completers who are in a related job 
as determined by the school, in the military, or in a school of higher 
education. Program reviews are done of 20 percent of the schools each 
year. During the reviews, a sample of employers is called and earnings 
information is collected from them. 

The vocational/technical schools use state-supported, computer- 
generated forms to submit the data on an annual basis. On-site program 
reviews are done to validate the data. For 1988-89, the rate of 
compliance was 84 percent. The accuracy of the data was more than 95 
percent. 

The Governor and the legislature are required to receive some of 
the reports; the information is used in considering legislation changes 
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and amendments. Secondary and postsecondary schools use the reports 
for program Improvement. Although it 1s public Information, the data 
are neither reported to students and their families nor generally made 
available from the schools. 

The state 1s In the process of piloting minimum competency testing 
for every high school vocational course In the state. A similar 
program 1s not planned for the postsecondary level. 

Proprietary Schools 

The only data that the proprietary schools submit to the state are 
enrollment numbers at the end of each year. The state tries to make 
one on-site visit to each school every year. DuHng the visit, faculty 
and student files are reviewed for items such as refunds for dropouts. 
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$wth Dakota 



A uniform data collection process is under development for public schools in South Dakota 
as part of a deal of more accountability for more budget flexibility. All aspects of programs 
arc included, such as < nrollments, placements, and completion data. Plans to conduct three- 
year follow ups are under way. Discussions have begun with proprietary schools concerning 
the need for standard performance measures. 



Organization of Postsecondarv Vocational Education 

The Vocational Education Office in the Division of Education 
oversees four vocational -technical institutes that enroll about 25,000 
students. Recent legislation established the postsecondary option 
program, which enables high school juniors and seniors to attend 
vocational/technical schools and receive dual credit. 

The proprietary schools in the state are licensed by the North 
Central Accreditation Agency. The state relies on this agency for 
assurance of the quality of education offered in these schools. 

After the 1989 Presidential Education Summit, the Governor held a 
series of diagnostic meetings on education in South Dakota. It was 
found that education was too rigidly academic in focus. As a result, 
it was recommended that the vocational education budget be increased at 
both the secondary and postsecondary level. The money is to be used 
toward increasing the number of programs offered, hiring faculty, and 
purchasing equipment. 

Public Schools 

Two years ago, four-year colleges, some of which award associate 
degrees, were granted budget flexibility in return for accountability. 
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A uniform data collection process is being developed that will 
encompass all aspects of the programs Involved. 

Vocational/technical Institutes provide enrollment and completion 
data to the state. A completer is a full -time student who has finished 
a program of study as evidenced by a certificate or diploma or through 
completion of a course sequence. About six months after completion, 
placement information 1s collected. The categories of placement 
utilized are: related/unrelated employment, continuing education, 
military service, seeking work, not employed, not continuing education, 
and not seeking work. Plans to conduct three-year follow ups are now 
under way. Earnings information 1s not collected by the state but some 
institutions do it on their own. 

Schools submit the placement, enrollment, and completion data in 
a written format to the state each year. The data are reviewed during 
product audits that are conducted every four years. The major purpose 
of the data 1s for management decision making, e.g., programs with low 
enrollments have been closed down. The information is used by the 
postsecondary schools, which must address identified areas of concern. 
It is not provided to secondary schools or the general public. 

Proprietary Schools 

The Association of Private Colleges is engaged in an ongoing 
dialogue with the Governor on the establishment of a long-term plan for 
higher education in the state. A major focus in this effort is the 
need for standard performance measures in postsecondary education. 
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Tennessee 

Tennessee uses school performance factors to allocate some supplemental funds to public 
schools over and above t>*se funding. :W perforraamje factors Include licensing test 
results, comptetkra and piscetmnt rates tylffltt and l»ee}, and alumni and employer 
wtteCMu A <*mipe^ each student individually 

over asteyear period; social security numbers are used. Proprietary schools annually report 
on number of compfetetsW 

of student complaints, documentation to advertising, enrollment agreements, and annual 
written reports. 



Organization of Postsecondarv Vocatio nal Education 

The Community Colleges Division of the Tennessee Board of Regents 
oversees 12 community colleges, 2 technical institutes, and 26 
vocational/technical schools. The community colleges and technical 
institutes award associate degrees and have a combined enrollment of 
60,000 students. The Tennessee Higher Education Commission is the 
licensing agency for 120 nondegree-granting private vocational 
education institutions. 

According to the Tennessee Student Assistance Corporation, the 
official student loan guarantee agency, Tennessee has a very low 
student loan default rate. The familiarity between lenders and 
borrowers was said to be the major reason for this low rate. 

Public Schools 

In addition to the regular school funding that Tennessee awarJs to 
all schools, the state provides some supplemental funding on the basis 
of each school's performance. Among the factors taken into 
consideration in the performance-based funding are licensing test 
results, completion rates, placement rates, alumni satisfaction, and 

154 

f l\ ' 



employer satisfaction. As a consequence of the performance-based 
supplemental funding, substantial information is collected at the state 
level. The statewide computerized student information system tracks 
each student individually over a six year period. Social security 
numbers are also in the system. 

Completion rates are calculated on an annual basis, with separate 
rates calculated by gender and race. A completer is one who has 
successfully finished a program and received an associate degree, 
certificate, or diploma. These rates are obtained directly from the 
computerized student information system. 

Schools conduct a survey of students two years after program 
completion. Survey topics include job placement, the relationship of 
employment to training, job satisfaction, and earnings level. 
Placement rates are calculated from this information; these are based 
on the number employed compared to the number eligible for employment. 
The number eligible is equal to the number of graduates minus those in 
military service and those continuing educatijn. A separate placement 
rate is calculated for completers who are in jobs related to their 
training; the school decides if the job is training related. These 
data are provided annually to the state on computer tapes. 

The Board of Regents receives information on the success rates on 
career licensing tests from the schools. Schools also provide 
information on entry and exit exams that must be given by every 
program. Licensing results and test scores are factors that are 
considered for performance-based funding. Performance data are needed 
by the legislative committee each January to make performance-based 
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funding allocations. Other recipients of the information are the 
Governor, the postsecondary schools, secondary schools, and the general 
public. The vocational/technical schools and community colleges use 
the information to maintain the quality of their programs. Schools 
must report to the state on how identified deficiencies are being 
remedied. 

The data also are used for management purposes such as resource 
allocation and revising the curriculum. Decisions on initiating new 
programs and discontinuing programs are based on performance measures 
and on reports from the Department of Employment Security of the number 
of vacancies per field in each district. In 1989, 46 programs were 
terminated. Additional information is obtained from a survey of 
employers on their satisfaction with the work performance of graduates. 

Proprietary Schools 

Information on number of completers is collected annually in the 
license application form. A completer is defined as one who has 
completed the course sequence for a program or who did not complete the 
program but obtained a training-related job. Some schools 
differentiate between full-time and part-time students. Placement rate 
data are collected annually as a part of the approval process. Annual 
site visits are conducted to ensure that schools are meeting their 
obligations under licensure rules. 

In October 1989, the Tennessee Higher Education Commission issued 
proposed revisions of the rules for private postsecondary education 
that are expected to be implemented in the near future. In some 
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background information issued along with the proposed revisions, the 
Commission stated: 

The most significant change is the tuition guaranty fund 
which will include assessment and collection of funds from 
private postsecondary vocational educational institutions 
which provide Instruction in Tennessee. Collections will be 
assessed Into a fund (similar to a self-insurance fund) from 
all such schools currently licensed and from private 
vocational education institutions which are accredited as 
degree granting Institutions and are exempted from the annual 
reporting or renewal process. 

Other changes in requirements are as follows: annual reporting of 
the number of formal written student complaints about substantive 
educational matters, institutional documentation of all advertising 
claims with information previously filed with the Commission, 
enrollment agreements between proprietary institutions and each student 
that list the projected courses necessary for the student to complete 
for attaining a specific credential, and written annual reports. 
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Texas 



Community college* in Tessas report on suttees rate* on Ucenstag tests and the number of 
students in the second year ot a two-year program, twt do not report completion rate*. 
They alio renort dat* on the Texas Academic Skills Program, wh(ch Consists of reading, 
wMn$ and mathematics tests; :i^joifcim eoUeeted hy the state from proprietary 
' ; schoob is that nmssary for licensing, whi<± v *easnre& = • :: . 



Organization of Postsecondarv Vocational Education 

The Commission on Higher Education 1s responsible for 100 
community colleges, enrolling approximately 346,000 students. Of the 
380 licensed private schools, 250 are members of the Association of 
Texas Private Schools. 

Public Schools 

Retention studies on the community colleges provide the Commission 
on Higher Education with Information on *M number of students who are 
in the second year of a two-year program; completion rates are not 
collected. Placement rates and earnings information are not collected 
by the Commission, but it does receive information on the success rates 
in career licensing tests from other state agencies. 

Every community college student must participate in the Texas 
Academic Skills Program, which consists of reading, writing, and math 
tests. Students who do not pass the tests receive remediation until 
they do. Data on the Academic Skills program are submitted in a 
written format to the state six times per year. They are reported to 
the legislature, school boards, superintendents, and public schools. 
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Proprietary Schools 

At the state level, the only Information collected 1s that 
necessary for licensing. No performance measures are Included. 

The Association of Texas Private Schools is just starting to 
collect information on completion and placement from its members. 
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Utah 



Community colleges in Utah submit data on enrollment, completion, placement, and 
programs offered. Although completion and placement rates are not computed, other 
measures collected include relationship of placement to training, earning levels, and gains 
in basic and job sidlls* One pilot prog^ of student 

: employment infonnation using social se<^rity norabers and unemployment wage data. 
Another is collecting additional employment toformation through a direct employer survey. 
Area vocational centers report data on completion, placement, and gains in basic, life, and 
job skills. Completion fates are not computed, but completion data are broken down by 
gender, race, and handicapping condition. Due to statutory limitations, the only 
performance moeures the smte coliects N)m proprietary schools are completion iin4 
placement data, and then only when these data are used in advertising, f 



Organization of Postsecondarv Vocational Education 

Utah's postsecondary vocational education system consists of five 
community colleges, five area vocational centers that also offer 
secondary level instruction, and one skills center serving 
disadvantaged adults. The Area Vocational Centers system is primarily 
designed for adult needs. It consists of 14 postsecondary institutions 
which include two community colleges, six junior colleges, five area 
vocational centers, and one center for disadvantaged adults. The 
institutions serve approximately 10,000 adults, some of whom receive 
secondary level instruction. The whole system is competency based. 
The community colleges and their vocational education programs serve 
approximately 34,000 students. 

The nonpublic sector includes 42 proprietary schools. The current 
student population in the proprietary schools is undetermined; however, 
the state's total vocational population is approximately 44,000 
students. All public colleges, except one research university, offer 
vocational programs. Among the proprietary schools, 23 are vocational 
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institutes, 1 awards the associate degree, and the other 18 are trade 
schools. 

The State Board of Regents is the governing board for community 
colleges. In addition, the Board has statutory authority over 
operation of proprietary schools and issues Certificates of 
Registration within legal guidelines. Except for the one skills center 
that is coordinated through higher education, Area Vocational Centers 
are governed by the Department of Vocational Education. 

Public Schools 

Community Colleges . Community colleges that pursue credit 
programs are eligible for state financial aid. Every five years the 
state evaluates all programs offered at the colleges. As part of the 
review, the schools submit data on enrollment, completion, placement, 
and programs offered. Evaluation reports are used for funding 
decisions. The state has defunded programs for low enrollment and Job 
demand. 

A completer is anyone awarded a certificate, diploma, or associate 
degree. Although completion rid placement rates are not computed, 
efforts have been made to collect other measures of performance, such 
as relationship of placement to training, earning levels, and gains in 
basic and job skills. Currently, gross quarterly earnings are the only 
income data collected on completers. The State Occupational 
Information Coordinating Committee (SOICC) is conducting a pilot 
program aimed at building a data base of student employment information 
to help identify placement by institution and program, additional 
earnings information, and to determine training relatedness. The pilot 
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has collected completers' social security numbers and has access to 
state wage data, welfare offices, and other job services agencies. 
However, without enabling legislation the confidentiality provisions 
will impede actual implementation of this program. 

Another ongoing pilot program is attempting to collect additional 
employment information on u sample of 100 completers by a direct 
employer survey. 

Several measures have been instituted to assess gains in basic and 
job skills. A Clerical Competency Program, designed to gauge clerical 
competencies by testing students on site under standardized work- 
setting conditions, was successfully piloted for three years and now is 
being partially implemented with state funding at school districts. 
Facilities and equipment are made available at the schools for students 
to test for a variety of other competencies. Further, competencies 
gained at high school level, for instance, auto mechanics and computer 
skills, are new transferable to community colleges. In addition, new 
vocational students in community colleges with certain work experiences 
will obtain exemption from course work by examination, and concurrent 
enrollment at the colleges have been made an option for the high school 
students. 

Site visits are made only for routine product and financial 
audits. Compliance with information requirements is high. 

Collected information such as enrollment and completion figures, 
programs offered, etc., is published annually in a "Fact Book" of state 
institutions distributed widely to the general public, lobbyists, as 
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well as the Governor's office, the legislature, and the State Office of 
Education. 

Several new policies have been formulated to enhance the quality 

of education at the community colleges: 

o Community colleges will soon convert to the competency-based 
system. 

o Program advisory committees with a primarily private sector 
membership have been established to help design curricula 
according to labor market needs. 

o Effective 1992, the Board of Regents will require every 
postsecondary institution to submit an annual report on its 
outcome assessment process. 

o The directors of area vocational centers were elevated to 
Dean level . 

o State specialists will be assigned to assist in the remedy of 
identified deficiencies in vocational programs placed on 
probation. 

Area Vocational Centers . The Area Vocational Centers use a 
competency-based system requiring submission of annual data on 
completion, placement, and gains in basic, life and job skills. A 
completer is defined as an individual who has demonstrated competency 
in a defined and approved instructional sequence that has been designed 
to prepare students for employment. Students complete programs at 
their own pace. Completion rates are not computed, but the completion 
data are broken down by gender, race, and handicapping condition, 
including the disadvantaged and those of limited-English language 
skills. Placement data are collected on an individual student basis 
using standardized demographic identifiers. Individual centers conduct 
surveys of the employers of completers to determine relationship of 
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placement to training, but the Information is not collected at state 
level . 

Assessment of gains in basic, life, and job skills is tailored to 
individual student needs and continues throughout the training program. 
Assessment instruments include the Test of Adult Basic Education 
(TABE). Entry and exit testing also is done. Analysis is done for 
both individual students and programs. 

All data are collected on standardized forms prepared by the Area 
Vocational Centers office, and on computer tapes. Site visits are 
conducted regularly during the year, during which product and financial 
audits are done. Compliance with information requirements is 
reportedly high, but data accuracy rate is not high, especially the 
student-specific information. Accuracy monitoring 1s a year-round 
activity as a result. Inaccurate data are returned to the centers for 
corrections or updates. 

A newly implemented policy, "Vocational Education Master Plan," 
requires joint secondary and postsecondary efforts in starting new 
vocational programs to avoid duplication of programs or curricula. 
Developed by the State Board of Regents and the Board of Education, the 
Master Plan prescribes a market-driven vocational education designed to 
train students in skills for actually existing jobs. 
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Proprietary Schools 

The state does not routinely collect measures of performance from 
proprietary schools due to statutory limitations. Completion and 
placement data are collected only from schools that are using the 
information for advertising purposes. Each school develops its own 
definition of a completer, which generally includes anyone who 
completed j program and was awarded a certificate, diploma, or any 
other credential. However, the Board establishes standards and 
criteria for awards of educational credentials. 

Site visits are conducted primarily to investigate public 
complaints such as false advertising, poor faculty performance, failure 
to recover tuition, etc., rather than to verify reported information. 
Schools found out of compliance must remedy identified deficiencies 
within a year or face stiff penalties that may include closure. Some 
proprietary schools have suffered this fate. An amended 1990 
legislative initiative broadens the Board of Regents' regulatory 
authority to include accredited and exempt schools. 

A new policy on proliferation of proprietary schools requires a 
feasibility study determining the labor market need for proposed 
programs before the school is approved to operate. In addition, 
schools setting up shop in Utah must have a physical presence in the 
state before a certificate of registration is issued. 
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Vermont 



The Vermont Technical College reports completion rates and completes a placement report 
containing placement rates, initial wage rates, and a list or employers who hired graduates. 
No performance data is collected from the other public and proprietary schools in Vermont 



Organization of Postsecondarv Vora^ onal Education 

The Division of Adult and Vocational Education in the Department 
of Education oversees 16 area vocational technical centers, 10 
proprietary schools, and 5 thirteenth year certificate programs. The 
State College System in Vermont includes Vermont Technical College and 
one community college that does not have any vocational education 
programs. The Vermont Technical College has about 700 PTE students, 
most of whom are full-time. 

Public Schools 

Vocational Technical Centers and Thirteent h Year Certificat e 
Programs. The quality of programs is primarily evaluated by 
approximately 30 technical committees which review curricula. Schools 
do not send completion, placement, or earnings information to the 
state. The state is considering student assessment based on competency 
per task. 

Vermont Technical College. The school receives various requests 
for completion rates from state agencies. These are calculated by 
comparing the number of students graduating with the enrollment on a 
program- by- program basis. Specifics on how these elements are counted 
depend on the directions with the data request. 
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The school prepares a placement report containing placement rates, 
Initial wage rates, and a listing of firms that have employed 
graduates. Students are considered placed 1f they are employed, joined 
the military, or transferred to a four-year college. The school tries 
to do a follow up on students every few years but it 1s not a 
formalized procedure. Follow up also may be done as a part of the 
accreditation process for a particular program. 

All incoming students in all state colleges in Vermont are tested 
in language arts and mathematics. Students who do not meet minimums 
are placed in developmental classes. 

The placement report Is sent annually to the State College System, 
key administrators at the college, and the admissions office. The 
report is disseminated in October and contains data on students who 
graduated in May. The Information 1s used informally within the 
school; program administrators can see which programs are having the 
most success and can share strategies with their colleagues. The 
information is not provided to secondary schools in any formal way but 
the college admissions office is in touch with secondary counselors. 
The college would give the information to students or potential 
applicants on request. 

Proprietary Schools 

Proprietary schools are subject to the same conditions as those 
for the Vocational Technical Centers and Thirteenth Year Certificate 
Programs . 
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Virginia 



Public schools in Virginia report the data needed to complete the IPEDS report A 

Number of graduates are reported by sex and race. Programs in proprietary schools that 
prepare students for unlicensed occupations report data on number of completers and 
placements each year as well as training-related placements and time needed for program 
•:||^P^ft>it " No performance standards are required for certification renewal. For 
proprietary schools preparing student for licensed occupations, data are reported to the 
appropriate licensing board. ; ; 



Organization of Pns tsecondarv Vocational Education 

The administration and monitoring of postsecondary vocational/ 
technical education in Virginia is spread across numerous agencies. 
The Community College System oversees 23 schools across the state. The 
Proprietary Schools Service of the State Department of Education 
oversees 140 schools with programs that do not require a license to 
practice. Proprietary schools or programs within schools for 
occupations that require a license are overseen by the board that 
provides the license. 

Public Schools 

The quantitative data submitted by the community colleges to the 
state are the information required to complete the IPEDS report for the 
National Center of Educational Statistics. Each school also must 
submit a narrative report to the state each year. Assessment issues 
are covered in these reports but each school addresses the issues in 
its own way. Much of the data related to placement, training-related 
placement, and wages are collected by the individual schools, but they 
are not reported to the state level and there are no data systems to 
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make 1t readily accessible. The state provides no standardized 
definitions for calculating completion or placement rates. 

The state compiles an awards booklet that shows the number of 
graduates broken out by factors such as sex and race from the data 
collected for IPEDS. Schools can access this Information 
electronically on terminals. 

Proprietary Schools 

Programs Preparing Unlicensed Students , For programs preparing 
students for occupations that do not require a license to enter, the 
proprietary schools are required to send information on tht number of 
completers and the number of placements to the division each year. The 
number of completers placed in positions related to training also is 
reported. The Individual schools make the determination of whether the 
placement 1s training-related. The schools also determine the period 
of time between completion and the collection of placement information, 
resulting in considerable variation. The state has attempted to 
calculate placement rates from the information provided under a 
standardized definition. The state's calculations are often 
considerably different from those done by the schools. For example, 
the schools may not include the following individuals in the count but 
the state does: those who did not want job placement, those who move 
from the area, and women who are about to have a baby. The state 
includes all program completers. 

Schools must include this information in their certification 
renewals but it is not required under any regulations. There are no 
standards of performance that must be met. 
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Programs Leading to a Licens e. Programs leading to a license 
include nursing, geriatric aide, flight instructor, barbering, 
hairdressing, cosmetology, truck driving, polygraph examiner, real 
estate, and private security guard. The individual schools report the 
information on these programs to the various boards involved. For 
example, the nursing board, which certifies programs in both public and 
private institutions, requests, but does not require, information on 
the number of admissions, graduates, and current enrollment. The 
nursing board also maintains data on licensing test results and can 
group the data by school . 

The data on admissions and graduates is verified every eight years 
during a site visit. Each year's submission also is compared to the 
submissions for previous years and anomalies are discussed wUh the 
reporting school. 

The data collected are reported to the Virginia Board of Nursing 
and also are mailed to individuals and organizations on the "public 
participation mailing list," which currently has about 2,000 names. 
There is no distribution of the report to the general public. 
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Washington 



Community colleges in Washington conduct ^If^MtioiA ittiiig the Washington State 
Higher Education Performance and Evaluation Pio^ measures reported 

are training-related placement, Ucense exam passage rites, ind student and employer ratings 
of effectiveness. Placement matching is done using student *oeW icctirily numbers and the 
state unemployment insurance data base, this will enable obttectkm of employment status 
and earnings, and trainingreiated placement determination. Both employer and student 
surveys are used. Completion rate is not deemed to be imports 
institutes are not participating in the performance evaluation program, but do report on 
number of enrollees and completers, they also surv^ students to obtain placement and 
training^ielated placement rates as well as earnings data> ^ are not 

required to submit any performance measures. 



Organization of Postsecondarv Vocational Education 

Within the state of Washington, there are 27 community colleges 
that graduate approximately 1,500 people per year in vocational/ 
technical programs. These schools are monitored at the state level by 
the Washington State Board for Community College Education. In 
addition, there are 5 vocational/technical institutes that offer 
programs to adults. The institutes are administered by the 
Superintendent of Public Instruction and offer preparatory training for 
entry into a trade, training to upgrade or retrain those air* Jy 
working, provide apprenticeships, and short-term training in consumer 
and homemaking skills. The Vocational Education Board oversees the 167 
proprietary schools and 1 nonprofit public trust vocational school. 
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Public Schools 

Community Colleges . In 1989, the state of Washington initiated 
the Washington State Higher Education Performance and Evaluation 
Program. The goal of the project is to provide information for self- 
evaluation by the institutions and for accountability. The program 
involves three measures of program effectiveness: training -related 
placement and success in licensure exams, employer ratings of 
effectiveness, and student ratings of effectiveness. 

The information for the measurement of effectiveness will come 
from a variety of sources. Information on placements will be obtained 
by matching community college students' social security numbers with 
social security numbers in the state wage data data base. The 
community colleges are required to provide the social security numbers 
of all students who have attended each institution and to indicate 
whether each completed the program or not. Arrangements also have been 
made to gain access to data from the state wage data bases of 
neighboring states. From this database it will be possible to 
determine whether the former student is currently employed or has been 
employed since leaving the school, hours employed, earnings, industry 
code, and county of employment. From the earnings data, an estimated 
annual wage will be calculated. The assessment of whether the 
employment is training-related will be made by cross-referencing 
Classification of Instructional Program (CIP) codes with industry SIC 
codes. 

A second source of information will be an employer survey. 
Employers will be identified from the matches with the state wage data 
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files. The employers will be asked about the degree to which they 
believe the community college system is producing students with 
competency in a variety of areas, and are able to apply their classroom 
training to the job situation. Approximately 3,000 employers will be 
surveyed each year. There will be a rotation by industry, so that any 
given industry will be surveyed only every third year. 

A third source of information will be the survey of a random 
sample of 800 students who left the community college system between 18 
months and 3 years previously. The students will be asked about the 
quality and relevance of their education. 

State wage data do not contain information on those who have gone 
on for more education, entered the military, are self-employed, or are 
employed by the Federal government. To develop estimates of the value 
of the vocational training received for these students and employers, 
a survey of a sample of those who do not appear in the state wage data 
data bases will be conducted every five years. Both completers and 
those who left before completing but are employed will be surveyed. 

The calculation of a completion rate doGS not receive much 
attention in Washington. About half of the students in the community 
college system obtain jobs before they complete their programs. It is 
felt these jobs are usually training-related and they are considered 
successful placements. Some of these students have completed all of 
their course work but have not yet paid a $10 fee for a diploma, so 
they are not technically classified as completers. 

As part of the comprehensive performance evaluation, more 
information on success with career licensing will be collected. 
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Currently, most of the career licensing boards have no data bases. In 
two years the state expects to be able to obtain career licensing 
Information on 7 of the 14 occupational areas that are licensed. 

The Information collected will be provided to the Higher Education 
Coordinating Board and will be provided to the schools. The Individual 
schools will be free to determine whether to disclose information to 
students or prospective students. The exact reporting format is under 
discussion. 

Procedures for monitoring compliance with this requirement or for 
assessing data accuracy have not been established. It is hoped that by 
using the state wage data, the need to establish elaborate and 
burdensome requirements for information collection and reporting by the 
schools will be avoided. 

Vocational Technical Institutes . These schools are not 
participating in the performance evaluation program, although there is 
discussion underway about using state wage data to assess placement and 
earnings of program completers. The schools currently collect 
information on the number of enrol lees and the number of completers and 
report it to the state. They also conduct a survey of completers to 
determine placement rate, which includes self -reports of training- 
related placement and earnings. 

The results of this annual survey are reported to the State Board 
of Vocational Education as part of an annual performance report. This 
report is provided to the Governor, the office of financial management, 
and the legislature. The report 1s not automatically distributed to 
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the schools, but 1s sent If requested. The Information also 1s 
available to the public on request. 

Proprietary Schools 

At the present time, proprietary schools are not required to 
submit performance measures to the state, and these schools do not 
participate In the performance evaluation program. There 1s discussion 
of Incorporating requirements for the collection and reporting of the 
measures listed in the Stafford loan regulations. Thirty percent of 
the licensed schools are accredited and therefore participate in the 
loan program. 

Previously, the state has not felt much need to become more 
systematically Involved in the assessment of the performance of these 
schools. Most are located in small communities; when graduating 
students from a particular school or program within a school aren't 
placed, "news travels fast" and students make responsible enrollment 
decisions. 
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West Virginia 



In the past, West Virginia collected performance outcome data from public school 
posisecondary graduates earning associate degrees through surveys asking about current 
placement, preparation for employment, present employer, job title, and relationship of 
employment to training. These surveys were dropped due to low response rates/ For 
proprietary schools, the state collects information on number of students completing, but 
not starting, programs by sex. they also collect the number of training-related job: 
placements. 



Organization of Postsecondarv Vocational Education 

Postsecondary vocational education in West Virginia 1s 
administered by the Division of Vocational, Technical and Adult 
Education within the State Department of Education. There are 11 
community colleges, enrolling approximately 25,000 students. There are 
52 proprietary schools; the state is in the process of collecting the 
number of students attending these schools. 

Public Schools 

In the past, follow-up surveys were made of associate degree 
graduates. The surveys, based on the old Vocational Education Data 
System (VEDS) form, asked about current placement, the name of the 
present employer when applicable, job title, and preparation for 
employment. Graduates also would indicate if they were employed in 
jobs related to their training; their responses were cross-checked with 
their job titles. Employers who were named by the graduates received 
a survey that asked about satisfaction with the employee and salary 
range. 



The surveys were done about every three years. Sometimes the 
state nailed them directly. Last time, the schools sent out the forms 
and the state paid for the postage. This change was made in an effort 
to Improve the response rate but it had little effect. The state does 
not expect to conduct these surveys In the near future unless It 1s 
mandated to do so. 

Programs are evaluated every four or five years. The information 
1s used by the Governor, the legislature, the State Board of Education, 
the post secondary schools, and the secondary schools, but not to 
students and their families. These evaluations are done to make 
management decisions; many programs have been terminated. 

Proprietary Schools 

The state collects the number of students completing, but not 
starting, programs. The number of women completers also is obtained. 
The number of students placed in jobs related to their field of study 
as determined by the schools is collected as well. 

The schools provide the data annually to the state in a written 
format. Data reporting occurs about four months after data collection. 
The information is used by the accrediting agency and the State 
Department of Education mainly to make management decisions. 
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Wisconsin 



Enrollment data and placement rates are reported by public schools each year in Wisconsin. 
Employment rates, training-related placements rate, and earnings also are reported. These : 
data are collected through a graduate survey, with a required survey completion rate. A 
1989 longitudinal study of the earnings of graduates of vocational, technical, and adult 
education examined the feasibility of using state income tax and wage reporting data as 
performance measures. Proprietary schools must indicate the number of students beginning 
and completing each program on their license renewal applications, with placement rates 
'. if they are used in advertising. ' '• 

Organization of Postsecondarv Vocational Education 

Wisconsin has 16 technical colleges that enroll 60,000 full time 
equivalent (FTE) students in degree programs and have additional 
students in nondegree programs. The technical colleges are overseen by 
the Board of Vocational, Technical and Adult Education. The state does 
not have community colleges. It does have two-year centers that feed 
into the University of Wisconsin system and do not grant degrees. 

The Educational Approval Board is the licensing agency for 
postsecondary proprietary schools except cosmetology and barbering. 
Technically, any school that has solicitors in Wisconsin or advertises 
in the state must have a license even if the school is not located in 
Wisconsin. Wisconsin licenses 85 schools, with about 65 to 70 of these 
are located in the state. These range from a neon sign-making school 
to r for-profit MBA program. Only about 10 to 15 of the schools are 
accredited, and therefore have students that are eligible for federal 
aid. 
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Public Schools 

Each year the number of graduates in full-time degree programs is 
collected by the state. Enrollment data for full-time and part-time 
students also is obtained. Completion rates are not calculated, but it 
is expected that half of the students in the full-time degree program 
will graduate each year. 

Placement rates are calculated based on the number of graduates 
divided by the number in the labor pool; those who are not looking for 
employment are excluded. Two additional calculations are made: 1) the 
percentage of those in the market who are employed, and 2) the 
percentage of those employed who are in jobs related to their training. 
Monthly salary rates also are requested of graduates. The state 
publishes salary ranges and median salaries for graduates who are 
employed full time in jobs related to their training. 

Placement and earnings information are collected in the standard 
survey that the schools send to all graduates six months after 
graduation. The schools must have an 80 percent response rate for the 
survey or show three attempts to contact nonrespondents. The graduates 
return the survey to the schools, which put the data on computer (tape, 
disk, or on-line). The data are submitted to the state, which compiles 
the data by program and produces a statewide report. The colleges may 
do their own report; the information may differ from the state report 
in that different methods may be used to do the calculations. Three- 
year and five-year follow ups are done on some classes. The career 
licensing boards are used as a resource for updated addresses for the 
follow-up surveys. 
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Success rates in career licensing tests are not required, but 
schools are encouraged to provide this information and an increasing 
number are doing so. Schools are asked to send as many measures of 
Impact as possible. Scores on basic skills tests are reported only for 
basic skills projects. 

The data are presented in a graduate follow-up report. From 
October through December, data are collected on students who graduated 
the previous December and May. The data are submitted to the state in 
January and the report becomes available in April. 

The state does not go on-site to monitor the accuracy of the data 
nor does it verify student follow-up information. Graduation rates 
over time are reviewed; programs that are expei iencing difficulties 
receive an in-depth evaluation. Programs have been discontinued and 
major curriculum modifications have occurred as a result of these 
efforts, e.g., last year a welding program was discontinued. 

The Governor and the legislature get copies of the report and ask 
questions about the information. The postsecondary schools and all 
secondary counselors receive copies. The report is available upon 
request to students and the general public. The report is primarily 
used for making management decisions. 

In 1989, the University of Wisconsin did a longitudinal ^tudy of 
the earnings of graduates of Vocational, Technical and Adult Education 
(VTAE) from the 1982-83 school year. The purpose of the study was to 
determine the feasibility of using state income tax information and the 
stata wage data reporting system instead of self -reporting techniques 
to provide performance measures of graduates. Data on the graduates 
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consisted of social security number, gender, age, program Instrw ;1onal 
area, length of instruction, district 1n which the Instruction was 
obtained, and whether or not financial aid was received. Student data 
were matched against Income records filed with the State Department of 
Revenue for the 1985 tax year. Unemployment compensation received and 
residence by zipcode also were collected for 1985. In addition, data 
from the 1988 wage reporting system, administered by the Department of 
Industry, Labor and Human Relations, were compared to 1982-83 student 
data. Researchers then developed a comparison group against which the 
1982-83 graduates could be measured. This group was based on 1985 
census data that was then adjusted to conform to the age and gender 
distribution of the 1982-83 graduates and to the ratio of earnings in 
the state compared to the nation as a whole. 

Proprietary Schools 

Proprietary schools must submit license renewal applications on a 
yearly basis. In the applications they must show the number of 
students who began and the number who completed each program during the 
previous year. Completion rates are not calculated but the number pf 
completers could be compared over time. 

Placement rates are not reported to the state. However, if a 
school makes claims in an advertisement, it must have records to back 
them up; these will be checked during an Inspection. The state law is 
very strict about reporting, therefore, most schools do not make 
claims. Schools are not required to report earnings information. The 
state tries to visit each school every two years or if problems arise. 
During an inspection, officials review student records, check 
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enrollments, make sure that refunds are made on time, and try to 
Interview students. 

No report is generated from the information that is collected. 
Some of the postsecondary schools make use of the data. The 
information is available for other audiences but there have not been 
any requests. Generally, if a program has a problem, it will show up 
sooner in student complaints than it will in the enrollment data. 
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Wyoming 



The only performance measures collected at the state level in Wyoming from community 
colleges are the measures from IPEDS reports. No performance measures are collected at 
the state level for proprietary schools. 



Organization of Postsecondarv Vocational Education 

Wyoming has 32 postsecondary and higher education institutions 
that include the University of Wyoming, seven community colleges 
offering associate degrees and a few vocational/technical programs, and 
17 proprietary schools. Public vocational training is primarily 
delivered at secondary level under 49 autonomous school districts. The 
state has no public postsecondary area vocational schools; as a result, 
regional vocational centers outside the state are utilized widely. 
Consequently, the state funds frequent travel of vocational counsellors 
to recruiting drives at vocational centers in neighboring Colorado, 
Idaho, Montana, and Utah. 

Wyoming's 17 proprietary schools offer vocational/technical 
programs to an estimated 2,260 students. The State Board of Education 
oversees the proprietary schools through statutory licensure authority. 
These students will become eligible for state financial aid effective 
October 1990 under a revised state law. 

Public Schools 

The State Commission on Community Colleges oversees the seven 
participating institutions. The only measures of performance collected 
at state level are the IPEDS reports, which are copied in the state 
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office before being forwarded to NCES. The completer is strictly one 
awarded the associate degree, diploma, or certificate of completion. 
Individual schools districts submit performance outcomes in vocational 
education collected through completer surveys conducted by instructors. 
The information collected identifies the local dropout rate and 
rationale, and the kinds of postsecondary schools or employment 
students enter after completion 

Historically, the legislature is reported to have taken very 
little interest in the need to improve vocational/technical education 
in general. This is reported to have been exacerbated by contemporary 
factors such as the state's depressed economic conditions mainly 
triggered by a declining oil market, and the resultant brain drain. 
Most recently the state has moved to eliminate the Division of 
Vocational Education altogether from the Department of Education. 
Previous considerations to transfer all oversight responsibility of 
vocational education to the Department of Labor effective July 1990 
also have been terminated. 

Proposed legislation promises to change this picture by upgrading 
quality and increasing availability of statewide vocational education 
at all levels. The bill seeks to establish an 11-member Vocational 
Education Task Force to evaluate current state policy of funding 
vocational programs through local school districts, investigate 
alternative funding mechanisms, and submit an interim study report with 
preemptive policy recommendations. The task force also would redefine 
Wyoming's vocational education needs to respond to existing business 
community needs, and in terms of emerging technologies and changing 
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labor market demands, thus steering away from the traditional 
agriculture-oriented curricula that historically have dominated the 
state's vocational training programs. The task force would further 
develop minimum education standards and objectives for vocational/ 
technical programs, establish accountability requirements with an 
assessment criteria, and articulate necessary state oversight. The 
task force membership would include representation from the business 
sector, legislature, state government, secondary and postsecondary 
vocational education instructors and administrators, parents of the 
students, and the at-large community. 

Reportedly because of the prevailing attitudes toward vocational 
education, the bill failed House introduction in the early 1990 
session. An amended version was to be introduced in the Senate. Its 
heaviest opposition reportedly emanates from the State Department of 
Education, which not only wishes to preserve the status quo and its 
vocational funding policy, but also contends that it is not a function 
of the legislature to conduct education studies of any kind. 

Proprietary Schoo ls 

Measures of performance are not collected at state level from 
proprietary schools as these are not a condition for licensing. 
However, license requirements stipulate that anyone who satisfactorily 
completes a training program be issued with a certificate. Study by 
correspondence is permitted. 
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Puerto Rico 



In Puerto Rico, state-level reporting from public postsectfndary schools Includes enrollment 
data and number of completers, broken down tv ^n«er» ^ of instituiton, and economic 
classifications. Completion and placement rates and relationship to training are not 
computer due to high unemployment mt& ^ has been 

instituted recently mandating schools to ^ 

graduates. voatiodttl/tohnicat instfwteVann^ placement, 
relationship of placement to training, and earnings levels, the only performance data 
collected from proprietary schools are enrollment and completion data at the beginning and 
end of each year. ^.&&V 



Organization of Postsecondarv Vocational Education 

The Commonwealth of Puerto Rico has an estimated 1,148 post- 
secondary institutions serving roughly 350,000 students. The 
nonpublic sector includes 24 two- and four-year colleges and 10 
specialized, serving 130,000 students with all overseen by the 
Commission on Higher Education. The 14 vocational/technical institutes 
are governed by the Department of Education's Office of Vocational 
Programs . 

There are 1,089 proprietary schools serving an estimated 200,000 
students. Those granting associate degrees are licensed through the 
Commission on Higher Education, and the rest by the Puerto Rico 
Approving Agency of Private Schools. 

Public Schools 

The funding breakdown for Puerto Rico's public sector institutions 
is one-third from Carl Perkins Act and two-thirds from state funds. 
The Commission on Higher Education annually collects enrollment data 
and total number of completers by programs, based upon tabulations of 
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responses to standard form questions it prepares for the schools. A 
completer is anyone awarded the associate degree with a 2.0 minimum 
cumulative grade point average. Completion rates are no,t calculated. 
Data are broken down by gender, and to facilitate other analyses, 
further by economic classification and type of college (public or 
private college). Placement rates and relationship to training are not 
assessed as the state's chronically high unemployment rate would render 
such data invalid. On-site verification is conducted every four years 
for accreditation purposes. 

The Commonwealth recently has instituted a variety of policy 
measures designed to improve and record performance outcomes. An 
outcome assessment mechanism established for the public sector 
mandates participating schools to submit five-year follow-up data of 
completers and graduates. Ongoing discussions with the accreditation 
agencies seek to get the state outcome assessment requirements 
incorporated into the accreditation process. Further, policy measures 
require schools to justify programs in terms of community and labor 
market needs. Declining enrollment has eliminated certain electronics 
and business education programs, and changing labor market needs have 
increased demand for computer and health technology programs such as 
physical therapy, radio therapy, etc. The schools also will subir.it 
data on retention trends. A standard definition of retention is being 
developed to include transfer students. The state has ascertained that 
70 percent of first year students in public institutions do not enter 
the second year. 
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These efforts are designed to serve management decision making as 
well as consumer rights purposes. Collected information is used to 
compile an annual report on postsecondary institutions showing 
enrollment and completion data by program. Copies go to the Governor, 
the legislature, participating institutions, and the general public. 

Measures o? performance are collected annually from the 14 
nondegree-granting, vocational/technical institutes through site visits 
by industrial coordinators. Collected information includes data on 
completion, placement, relationship of placement to training, and 
earnings levels. No site visits are conducted for data accuracy 
verification due to staff shortage. 

A completer is dnyone who has finished required training hours and 
internship for a program and subsequently was awarded a certificate. 
The industrial coordinator is also responsible for placement, in 
consultation with the State Department of Labor (DOL). Hence, the 
completion and placement data are broken down in terms of services 
completers will need. A state DOL representative participates in the 
placement effort by interviewing completers and matching them with 
training-related jobs listed with the DOL. The highest placement 
levels occur for completers of technical, marketing, and distribution 
programs. The relationship of placement to training and initial wage 
levels are determined through follow-up studies conducted one and two 
years after placement. 

The analyses of collected information are disseminated in an 
annual report prepared jointly with the Puerto Rico Occupational 
Information Coordinating Committee and the Department of Labor. 
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New measures are under consideration to enhance the quality of 
public vocational training 1n Puerto Rico. A current policy proposal 
seeks to integrate academic basic skills 1n all vocational/technical 
programs and later Include these in outcome assessment. The proposal 
seeks to hire occupational counselors for all vocational centers to 
help assess students' ability to benefit from self- selected programs. 
Technical programs currently assess basic skills at entry and have 
prepared remedial courses in Spanish, English, mathematics, physics, 
and communications. Plans also are underway to collect annual data on 
success rates in career licensing tests as a measure of program 
quality. Other plans Include: the establishment of an academic track 
professional development unit to improve teacher competencies, 
curricula revisions responding to the needs of emerging technologies, 
and regular updates of existing facilities and equipment. 

Proprietary Schools 

Only enrollment and completion data early in the year and at year 
end are collected from the proprietary schools, using a standard form 
prepared by oversight agencies. A completer is the recipient of an 
associate degree, or anyone £ha either obtained a certificate in an 
occupational area or completed all training hours for a given program. 
Collected Information 1s broken down by gender. 

Site visits for the proprietary schools are conducted every four 
years, during which all operations of the institution are reviewed for 
license renewal. Additional visits are made promptly to Investigate 
student grievances. Schools failing to submit required information 
also undergo investigative reviews. Noncompliance with reporting 
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information is somewhat high. Such schools generally have expired 
certification and consequently missed the state mailing list for 
reporting forms. Data inaccuracies are remedied by telephone and 
written memos. Schools are mostly cooperative. Statutory provisions 
mandate that all deficiencies be remedied in six months or the violator 
faces loss of license. 

The information collection effort is aimed at both management 
decisions and consumer rights purposes. The State Office of 
Information and Statistics uses the data to compile two statistical 
reports: the annual report on private school enrollment, published 
early in the year, and the Stat istical Annual Repor t, which includes 
enrollment and completion data for both private and public schools and 
is released at the end of the year. The state also publishes an annual 
directory of licensed schools and programs offered. In addition, a 
public affairs office answers public inquiries on the status of any 
proprietary school. All compiled reports are distributed to the 
Governor, the legislature, and the general public. 

Site visits to degree-granting proprietary schools are conducted 
primarily for program evaluation. Student complaints against a school 
generate additional investigative visits during the review cycle. 
Programs are evaluated on individual basis. New programs are evaluated 
twice: before introduction and four years later. The program-specific 
review process is designed to correct all deficiencies at once. It. 
therefore, covers diverse areas such as viability data, student profile 
after graduation, curriculum, faculty qualifications, facilities and 
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equipment, and placement data when used in school recruiting 
activities. 

Noncompliance with information requirements is a regular 
occurrence among the institutions. The highest rate is found in the 
area of facilities and equipment (I.e., libraries, technical 
appliances, etc.), curriculum revision, and faculty qualifications. 
Several programs have been closed down for violation of these and other 
licensure requirements. 

Evaluative reports based on site visit reviews are supplied to 
participating schools, who in turn use them in marketing activities. 
Truth in advertising 1s strictly monitored, and the state maintains an 
advertising tracking office for this purpose. False advertising leads 
to prosecution and eventual loss of license. This punitive outcome was 
said to occur frequently. 

Students in proprietary schools are eligible for only portions of 
state financial aid. These include 50 percent of funds in th* Puerto 
Rico Education Fund earmarked for public and private institutions, and 
15.6 percent of the legislated scholarships. 

At the moment the state has no proposed or pending legislation 
addressing the issue of consumer rights information in postsecondary 
education. 
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IV. INFORMATION MATRIX 

To provide the results of the overview in a more accessible form 
and to make patterns and comparisons more readily identifiable, the key 
information elements are arrayed in a series of matrices. The matrices 
are designed to complement the narrative summaries and present 
Information consistent with those summaries. 

Column Definitions 
In developing the matrices, project staff sought to identify the 
most critical information elements to serve as column headings. A 
procedure for coding the states on each column was developed to ensure 
accurate interpretation of the information presented. A summary of the 
coding procedure for each column is presented below. 

Outcome Measures Collected at State Level 

This section of the matrix refers to efforts to collect specific 
outcome measures about training program leavers. Both efforts at the 
state level to collect the information directly, or efforts resulting 
from requirements that the information be collected by the schools or 
instructors and forwarded to the state are included. Information 
efforts conducted by schools on their own initiative and never sent to 
the state, or sent to the state on the school's initiative are not 
included. 

Completion. This column indicates whether information on the 
number of completers is collected at the state level. A state is coded 
as "Yes " if any counts of students completing a program are required to 
be reported to the state. A state could be coded as "Yes" regardless 
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of the definition of a completer used and whether a standardized 
definition was used statewide. A state also could be coded as "Yes" 
even 1f the completion counts were not used to compute a completion 
rate. 

PI acement . This column indicates whether information on the 
number of persons formerly attending postsecondary vocational education 
programs who are subsequently employed is collected at the state level. 
As with the completion counts, a state is coded as "Yes" if any counts 
of job attainment are reported to the state, regardless of the 
definition of a placement or of the categories of former students for 
whom counts are obtained. 

Training Related Placement . This column shows whether the report 
on placement information contains any indication of whether the job 
attained 1s related to the content of the vocational training received. 
Any effort, without judgment by project staff of its validity, was 
coded as "Yes." 

Earnings Information . Any effort to collect information on the 
earnings of individuals who had attended the training program was coded 
as "Yes." Either efforts to measure initial wage levels or follow-up 
information at some later point, such as 5 years after leaving the 
training program, would be included. A "Yes" also is given regardless 
of the source of the information, whether it be a survey of program 
leavers or employers, or the use of third-party information. 
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Licensing . This column indicates whether information on the 
performance of completers and possibly other program attendees is 
collected for programs in the licensed trades. This includes whether 
information is sent directly from the licensing board to the relevant 
state agency, or whether the information is reported to the schools and 
then forwarded. 

Gains in Ba sic Skills or Job Knowledge . If efforts are made to 
assess gains in basic skills or in knowledge specifically related to 
the occupation for which students are being trained, a "Yes" is entered 
in this column. An assessment must be made at two points in time 
during a student's time in the training program to be considered an 
assessment of gains. The use of basic skills tests by proprietary 
schools at program entry only to establish the potential for benefiting 
from the program is not included. 

Use of External Data . This column is marked "Yes" if external 
data sources are used by the schools or state to provide any 
information on placement, on whether the placement is training related, 
or on earnings. State wage data and data from IRS are two of the 
external sources used by the states. 

Fol low-Up Survey . This column refers to efforts to gain 
information from training program completers (and sometimes leavers who 
did not complete the program) or their employers through a survey. 
Included are sample surveys or attempts at a census, and all surveys 
are included regardless of the interval after the students leave the 
program. The content of the surveys usually include information on 
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placement, whether the placement 1s training related, and student and 
employer satisfaction. 

Statewide Standard Definition . A "Yes" 1n this column means that 
the definition of at least one of the performance measures 1s 
established by the state oversight agency to be used consistently 
throughout the state. Evidence of a standard definition includes the 
use of a state data form, publication of a definition In state 
documents, or Indications from respondents that Information supplied 1s 
consistent throughout the state. 

Other Characteristics of Outcome Measurement 

Data Uses . Two types of data uses are entered on the matrix: 
management and consumer rights. Management refers to the utilization 
of the data for state administrative purposes such as decisions about 
the funding or defunding of schools and programs, the licensing of 
schools, or preparing reports for the state legislature. Consumer 
rights uses indicate that some report that would allow comparisons of 
the outcome' data across schools is available to potential students, 
their parents, and the general public. This would include having the 
information distributed to secondary schools, to libraries, or 
available upon request. 

Data Verification . This column is marked "Yes" when independent 
data collection is regularly done by the state or when third-party data 
are used as a major information source. A site visit to a school to 
review the data is shown as a "No" unless additional data are collected 
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during the site visit. Infrequent verification efforts by the state, 
e.g., once every five years, also are marked as "No." 

Recently Placed Into Effect . A "Yes" is entered if any of the 
outcome measures were added or changed within the last two years. 

Pending Changes . Possible changes that have not yet gone into 
effect are described in this column. These include changes being 
discussed by the state legislature or oversight agency, legislative 
changes for which rules or regulations are being written, changes that 
have been developed but not yet implemented, and new student tracking 
systems that are under development. 

The Matrices 

The indicators selected to be included in the matrix wer-e those 
judged to provide the best descriptive information about the practices 
and policies in program outcome measurement in each state. The number 
of elements that could be included was constrained by the desire to 
concentrate the presentation of information into one location. 

The constraints of size in the presentation of the matrix versus 
the number of elements selected to be presented resulted in the 
decision to prepare one matrix for public institutions, and one for 
proprietary institutions. Because the outcome measurement policies for 
community colleges and vocational/technical schools was similar in most 
states, these two components have been combined. For each column of 
the matrix, if either the community colleges or vocational/technical 
schools could be coded as a "Yes" a yes is entered. The state 
narratives presented above provide more detailed information about any 
differences between these components. 
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MATRIX OF INFORMATION ON STATE ACTIVITIES 
IN THE COLLECTION AND USE OF OUTCOME MEASURES 
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